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PREFACE 


For  the  eighteenth  year,  the  Research  and  Theory  Division  of  the  Association  for 
Educational  Communications  and  Technology  (AECT)  is  sponsoring  the  publications  of  these 
Proceedings.  Papers  published  in  this  volume  were  presented  at  the  national  AECT  Convention  in 
Indianapolis,  IN.  A limited  quantity  of  this  volume  were  printed  and  sold.  It  is  also  available  on 
microfiche  through  the  Educational  Resources  Information  Clearinghouse  (ERIC)  system. 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the  AECT 
convention  and  included  in  this  Proceedings  were  subjected  to  a rigorous  blind  reviewing  process. 
All  references  to  author  were  removed  from  proposals  before  they  were  submitted  to  referees  for 
review.  Approximately  fifty  percent  of  the  manuscripts  submitted  for  consideration  were  selected 
for  presentation  at  the  Convention  and  for  publication  in  these  Proceedings.  The  papers 
contained  in  this  document  represent  some  of  the  most  current  thinking  in  educational 
communications  and  technology. 

A selected  number  of  development  pages,  sponsored  by  the  Division  for  Instructional 
Development  (DID),  are  included  in  this  Proceedings.  The  most  important  instructional 
development  papers  were  selected  by  the  DID  program  chairs  for  publication. 

This  volume  contains  two  indexes.  The  first  is  an  author  index;  the  second  is  a descriptor 
index.  The  index  for  volumes  1-6  (1979-84)  is  included  in  the  1986  Proceedings,  and  the  index 
for  volumes  7-10  is  in  the  1988  Proceedings.  After  1988,  each  volume  contains  indexes  for  that 
year  only. 


M.R.  Simonson 
Editor 
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and  Technology 


What  is  AECT? 

AECT  is  the  only  national,  professional  association 
dedicated  to  the  improvement  of  instruction  through  the 
effective  use  of  media  and  technology.  AECT  assists 
its  members  in  using  technology  in  their  jobs  and  to 
enhance  the  learning  process. 

Who  belongs  to  AECT? 

□ Media  specialists 

□ Educators 

□ Librarians 

□ Instructional  designers 

□ Corporate/military  trainers 

□ Learning  rosource  specialists 

□ Curriculum  developers 

□ Television  producers  and  directors 

□ Communications  specialists 

□ Education  administrators 

□ Others  who  require  expertise  in  instructional 
technology 

What  are  AECT  members  involved  in? 

□ Hypermedia 

□ Interactive  Video 

□ CD-ROM 

□ CDI 

□ Teleconferencing 

□ Film  and  Video  Utilization 

□ Telecommunications 

□ Computer  software  and  hardware 

□ Projection/presentation  products 

□ Intelligent  tutoring  systems 

□ Videodiscs 

□ Distance  learning 

□ And  morel 

AECTs  publications,  convention,  trade  show,  and 
conferences  present  the  leading  edge  on  research  and 
practical  applications  for  these  and  other  technologies. 

AECT  History 

Unlike  some  other  special-interest  technology 
organizations,  AECT  has  a long  history  with  over  70 
years  in  educational  technology.  We’ve  grown  up  with 
technology,  advocating  its  integration  into  education 
from  films  to  integration  into  education  from  films  to 
overheads  to  interactive  video  and  hypermedia. 


AECT  began  as  the  Department  of  Visual  Instruction  at 
the  National  Education  Association  in  1923,  in  the  days 
when  visual  aids  consisted  of  films  and  slides.  In  1947, 
as  educators  were  adapting  technology  used  to  train 
World  War  II  service  personnel  for  the  classroom,  the 
name  of  the  organization  became  the  Department  of 
Visual  Instruction  (DAVI).  Twelve  years  later,  DAVI 
became  an  affiliate  of  the  NEA  and  finally  the 
autonomous  association,  AECT,  in  1974. 

Today,  AECT  keeps  an  eye  on  the  future  of  instructional 
technology  while  assisting  educators  with  the  changes 
and  challenges  that  face  them  now.  AECT  members, 
now  numbering  4,500,  are  professionals  devoted  to 
quality  education.  They  care  about  doing  their  jobs 
better  and  want  to  embrace  new  methods,  new 
equipment,  and  new  techniques  that  assist  learning. 

Membership  in  AECT  increases  your  effectiveness,  your 
expertise,  and  your  skills.  These  qualities  in  turn 
enhance  your  professional  image  and  earning  potential. 

AECT  Affiliates 

AECT  has  47  state  and  16  regional  and  national 
affiliates,  and  has  recently  established  several  chapters 
surrounding  major  universities  and  metropolitan  areas. 
These  affiliated  organizations  add  a localized  dimension 
to  your  AECT  membership  and  allow  for  more 
interaction  among  your  colleagues.  For  more  details 
on  chapters  and  affiliates  in  your  area,  contact  the 
AECT  National  Office. 

AECT  National  Convention  and  InCITE 
Exposition 

Each  year,  AECT  brings  top  speakers  to  exciting 
locations,  and  presents  over  300  sessions  and  special 
events  to  provide  the  best  training  available  in  the  use 
of  media  in  education  and  Instruction.  The  convention 
features  the  InCITE  Exposition,  the  first  trade  show 
created  exclusively  for  instructional  technology 
products.  At  InCITE,  you’ll  see  computers,  learning 
systems,  software,  interactive  multimedia,  audiovisual 
products,  films  and  videotapes,  projectors  and 
presentation  products,  video  equipment,  accessories, 
and  more.  The  convention  offers  tracks  of  sessions 
focusing  on  specific  interest  areas  surrounding  AECTs 
nine  divisions  and  other  special  interests.  The 
convention  has  featured  a Hypermedia  Strand  and  a 
Total  Quality  Management  Track.  In  addition,  intensive 
full  and  half-day  workshops  are  offered  for  in-depth 
training  on  the  latest  technology  applications  for 
education. 

Research  and  Theory  Division  ( RTD)  improves 
the  design,  execution,  utilization,  evaluation,  and 
dissemination  of  educational  technology  research  and 
theory;  advises  educators  on  using  research  results. 
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The  problem  with  pictures  is  that  they  are  largely  contextually  defined  instead  of  being  structurally  defined. 

That  is,  it  is  much  more  common  for  a person  to  describe  a picture  in  terms  of  the  contextual  objects  (people,  places,  or 
things)  within  the  picture  than  in  terms  of  the  visual  elements  that  make-up  the  picture  (circles,  squares,  points,  lines, 
colors  and  shading).  One  might  say,  “Of  course!  It  would  be  too  cumbersome  to  describe  a picture  in  those  terms 
because  they  are  not  specific  enough!”.  This  statement  is  true  if  you  are  only  considering  the  contextual  meaning  of  a 
picture.  Circles,  lines,  and  points,  have  no  contextual  meaning  of  an  by  themselves.  However,  if  an  artist  wishes  to 
create  or  modify  a picture,  the  only  elements  available  for  this  purpose  are  lines,  circles,  squares,  color  and  shading. 

There  are  no  art  brushes  that  paint  faces  automatically  such  that  when  the  brush  is  dragged  across  the  canvas  a face 
appears.  Instead  the  artist  must  vary  lines,  shapes  and  points,  combined  with  shading  and  textures,  in  such  a way  as  to 
produce  contextually  recognizable  elements  in  the  picture.  The  question  is  then,  how  does  and  artist  know  how  to  vary 
these  structural  elements  in  such  a way  as  to  produce  the  desired  contextual  meaning  of  a picture  if  these  elements  have 
no  contextual  meaning  of  and  by  themselves?  Even  more  problematic  is  how  an  instructional  designer  specifies  a 
necessary  picture  to  an  artist  such  that  the  appropriate  contextual  meaning  is  communicated  from  the  picture  the  artist 
creates. 

In  an  instructional  message  there  are  often  subtle,  yet  critical,  aspects  that  must  also  be  emphasized  in 
supportive  graphics  or  images.  The  instructional  designer  must  be  able  to  describe  the  contextual  emphasis  such  that  an 
artist  or  graphic  designer  will  be  able  to  manipulate  the  structural  elements  of  the  picture  in  a corresponding  fashion.  The 
goal  is  to  make  the  structural  emphasis  congruent  with  the  contextual  emphasis.  The  emphasis  of  the  contextual 
message  and  the  emphasis  of  the  structure  of  a picture  is  referred  to  in  this  paper  as  contextual  dominance  and  structural 
dominance  respectively. 

In  an  instructional  message  the  contextual  dominance  is  most  often  conveyed  in  the  form  of  printed  or  spoken 
sentences.  Within  any  sentence  used  in  conjunction  with  a picture  are  nouns  or  phrases  that  directly  relate  to  contextual 
elements  within  the  picture.  For  instance  if  we  are  presented  a picture  and  heard  “The  man  walked  through  the  door.”  we 
would  expect  to  identify  a man  and  a door  in  the  picture.  Both  “man”  and  “door”  may  be  called  referents  in  the  sentence 
since  they  refer  to  objects  perceptible  in  the  picture.  The  study  described  herein  varied  the  dominance  of  referents  used  in 
a number  of  sentences  and  compared  the  patterns  of  subsequent  observations  of  1 5 pictures.  The  goal  was  to  identify 
structural  and/or  contextual  elements  that  stimulated  consistent  patterns  of  observation.  A brief  discussion  of  the 
literature,  the  methodology  used,  results  for  two  pictures,  and  a summary  of  conclusions  follows. 


Synthesis  of  the  Literature 

Discussing  how  someone  views  and  image  depends  also  on  describing  the  image  itself.  Most  of  the  research  to 
date  use  contextual  definitions  to  describe  the  images  and  not  structural  ones;  but,  there  have  been  some  attempts  to 
quantify  the  structure  in  terms  of  variations  of  color,  complexity  and  layout. 

Color,  Complexity,  and  Layout 

In  general  color  has  been  found  to  be  helpful  when  utilized  to  emphasize  dominant  contextual  features,  but  it  is 
a distracter  when  not  relevant  to  context  (Luder  & Barber,  1984;  Yarbus,  1967;  Reid  & Miller,  1980)  For  instance, 

Luder  and  Barber  found  that  color  could  be  utilized  to  highlight  specific  elements  within  a complex  display,  while  Reid 
and  Miller  found  that  full  color  illustrations  of  anatomical  drawings  presented  too  many  distracters  from  what  was 
considered  important  in  the  drawing. 

Complexity  and  picture. layout  has  been  found  to  elicit  different  viewing  strategies  among  students. 

Specifically,  Malcolm  Fleming  utilized  four  "layouts",  one  with  a caption  preceding  an  illustration  and  another  with  it 
following.  Each  of  these  layouts  also  varied  by  complexity.  Utilizing  eye  movement  as  a dependent  variable,  he  found 
that  gender  and  previous  experience  with  the  information  were  strong  factors  as  to  which  strategy  to  process  the 
information  was  chosen-  image  first  or  caption  first  (Fleming,  1984). 

Another  attribute,  which  relates  to  complexity,  is  that  of  the  "degree  of  realism"  to  which  an  image  may  be 
attributed.  The  "degree"  represents  a continuum  from  concrete,  or  realistic,  to  abstract,  or  non-realistic.  Gavriel  Salomon 
describes  this  continuum  operationally  in  terms  of  the  degree  of  "coding"  one  must  do  when  encountering  a visual 
representation  of  something. 
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"Certain  representations  appear  to  be  more  "realistic"  because  their  symbolic  form  comes  closer  to  the 
way  users  represent  the  depicted  entity  to  themselves.  The  less  recoding  something  requires,  the 
mentally  "easier"  it  is  to  experience  and  the  more  "real"  it  appears. " (Salomon,  1981  pg.  201 ) 

Relative  to  this  "concrete  to  abstract  continuum",  concrete  images  are  remembered  better  than  abstract  images  (Paivio, 
1983;  Winn,  1982;  Findahl  & Hoijer,  1976;  Wolf,  1970).  On  the  other  hand,  abstract  graphics  have  been  found  to  be 
more  successful  in  educational  contexts  due  to  fewer  structural  elements  (Heuvelman,  1987). 

Visual  Design  of  an  Image 

One  aspect  left  largely  unstudied  in  most  of  the  research  in  image  utilization  is  that  of  the  visual  design  of  the 
stimuli  being  used.  Visual  design  here  refers  to  the  structural  relationships  of  visual  elements  within  an  image.  Layout, 
mentioned  earlier  in  the  Fleming  study  of  complexity,  is  the  closest  variable  this  researcher  found  in  the  literature  to  that 
of  visual  design.  Nesbit  captured  this  very  common  problem  in  the  following  statement. 

"Little  attempt  was  made  to  examine  the  design  qualities  of  the  picture  itself  or  the  component  cues  of 
the  picture  in  terms  of  learning  theory.  All  modes  of  pictorial  representation  were  considered  as 
infinitely  large  masses  of  stimuli  and  examined  as  such...  no  attention  was  given  to  isolating  those 
elements  which  make  an  instructionally  effective  visual. " (Nesbit,  197a  pg.  496) 

The  variable  of  style  was  of  interest  to  Molner.  He  believed  that  traditional  definitions  of  style,  usually  treated  as  purely 
affective  and  subjective,  were  identifiable  in  terms  of  structural  attributes  of  an  image.  His  eye  movement  study  of 
Renaissance  and  Baroque  paintings  illustrates  that  the  structural  attributes  of  paintings  may  be  utilized  in  such  a stylistic 
manner  that  it  can  influence  the  viewer  to  scan  an  image  at  a certain  speed  and  focus.  He  found  that  art  of  the 
Renaissance  was  viewed  with  large  and  slow  eye  movements,  while  Baroque  art  produced  denser  and  shorter  eye 
movements  (Molner,  1981). 

Image  and  Language  Processing  in  a Context 

Utilizing  a "dual-coding  information  processing"  model  of  learning,  Kozma  points  out  that  if  information  in 
long-term  memory  may  be  stored,  not  only  semantically  but  pictorially  as  well,  then  images  can  be  retrieved  into  short- 
term memory  in  response  to  either  nonverbal  or  verbal  stimuli.  He  summarizes  the  research  by  stating  that  if  the  same 
information  is  stored  both  pictorially  and  verbally,  it  is  more  likely  to  be  retrieved  (Kozma,  1986). 

Just  as  printed  text  under  an  image  provides  contextual  information  utilized  to  view  an  image,  so  does  spoken 
text  during  or  immediately  preceding  an  image  affect  the  processing  of  that  image.  Many  media  today  entail  the  learner 
processing  audible  textual  information  and  visual  information  arriving  at  the  same  time.  Very  few  studies  have  examined 
the  intricate  interactions  involving  the  chunking,  sequencing,  and  pacing  of  these  dual-channel  stimuli.  But,  it  has  been 
confirmed  that  the  majority  of  the  contextual  effect  is  provided  by  the  verbal  text,  and  when  verbal  contextual  cues  are 
presented  prior  to  viewing  an  image,  a semantic  attention  to  the  image  is  evoked  (Koroscik,  Desmond,  & Brandon, 

1985). 

This  strategic  focus  supports  previous  prescriptive  conclusions  relating  to  color,  complexity,  and  the  schematic 
design  of  an  image.  Both  point  to  contextual  cues  dictating  structural  prescriptions  in  the  visual  channel  and  specific 
reliance  on  the  verbal  channel  for  being  the  major  carrier  of  contextual  information. 

Koroscik,  Desmond,  and  Brandon  examined  this  relationship  among  structural,  semantic,  and  verbal  contextual 
information.  They  began  the  study  with  the  following  hypotheses:  It  was  speculated  that  the  encoding  and  retention  of 
art  is  subject  to  the  type  of  contextual  information  given  to  viewers  at  presentation.  Verbal  labels  with  literal  references 
to  the  objects,  persons,  or  events  depicted  in  an  artwork  ought  to  evoke  semantic  encodings  that  differ  from  those 
generated  in  response  to  verbal  references  pertaining  to  the  work's  expressive  qualities  and/or  other  non-literal  aspects  of 
the  depicted  content.  (Koroscik,  1984  pg.  332) 

By  presenting  first  a verbal  contextual  cue  and  then  presenting  an  image,  they  were  able  to  measure  the  effect  of 
the  context  on  retention  of  the  desired  message.  Since  the  contextual  cues  referred  literally  to  structural  elements  within 
the  images,  some  information  was  also  gained  about  the  degree  of  distraction  of  "non-pertinent"  structural  elements. 
Results  also  indicated  that  accurate  interpretation  of  meaning  was  a function  of  the  level  of  abstraction  that  characterized 
each  artwork  and  of  the  type  of  contextual  information  given  at  input  (Koroscik,  1 984). 
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These  results  offer  very  few  prescriptions  to  a message  designer  other  than  a sense  that  context  and  the  structure 
of  an  image  may  somehow  be  interdependent.  On  one  hand  structural  concerns  are  important  for  immediate  interpretation 
and  long-term  memory,  yet  on  the  other  there  is,  at  some  point,  a shift  of  focus  to  contextual  aspects  of  an  image. 

Which  is  dominant,  structure  or  context?  What  is  the  relationship  between  the  abstract-to-concrete  word  continuum  and 
an  abstract-to-concrete  image  continuum?  The  concept  of  congruency  begins  to  explore  the  relationships  posed  by  these 
questions. 

Congruency  Between  Images  and  Language 

In  his  review  of  the  contributions  of  eye  movement  studies  to  research,  Marschalek  states: 

..thus  the  compositional  structure  affects  perception  most  dramatically  when  structure  and  meaning  are 
united  within  the  same  areas  of  the  picture.  ( Marschalek , 1986  pg.  135) 

This  succinct  statement  encapsulates  the  idea  of  congruency  between  words  and  images.  The  concept  of  congruency 
between  the  structural  components  of  an  image  and  the  context  presented  is  of  extreme  importance  to  the  message 
designer.  Congruency  deals  with  the  basic  problem  of  linking  words  and  images  together.  Many  researchers  have  found 
that  when  structural  features  and  contextual  features  are  congruent,  then  attention  to  the  message  is  maximized 
(Heuvelman,  1987;  Hsia,  1977;  Marschalek,  1986;  Miller,  1982;  Nodine,  1982;  Wember,  1976) 

A growing  number  of  cognitive  scientists  are  specifically  looking  at  the  relationship  between  the  cognitive  processing  of 
words  and  the  processing  of  images.  A Dual-Coding  Theory  was  developed  by  Paivio  which  stated  that:  . 

nThe  verbalization  of  a picture’s  features  increases  the  probability  that  two  codes  are  activated  in  the 
formation  of  stimulus  memories.  The  argument  is  that  sensory  features  of  pictures  are  stored  in 
imagery  codes,  while  the  products  of  verbalization  are  retained  as  verbal  or  linguistic  codes. " ( Madigan , 
1982 , pg.  80) 

This  statement  is  contrary  to  a "common  code"  view  that  says  pictures  somehow  possess  a faster  access  than  words  to  a 
common  conceptual  system.  The  dual-coding  view,  on  the  other  hand,  sees  picture-word  latency  differences  as  stemming 
from  a time  consuming  translation  from  one  symbolic  code  to  another  and  that  semantic  information  required  in  the 
decision  task  is  typically  stored  nonverbally.  Numbers  of  other  researchers  have  been  testing  similar  concepts  and  have 
arrived  at  some  meaningful  conclusions.  Segal  and  Fusella  (1970)  found,  by  using  interference  tests,  that  cognitive 
processing  is  modality  (visual  or  audio)  specific,  i.e.,  the  brain  operates  with  a separate  processing  system  for  each 
modality.  Nugent  (1982)  and  Wickens  (1984)  found  that  "learners  process  pictorial  and  linguistic  information  through 
functionally  independent,  though  interconnected,  cognitive  systems.” 

Whenever  one  deals  with  issues  of  context,  they  are  subject  to  personal  interpretation  as  to  their  meaning, 
importance,  and  dominance.  For  an  individual,  attribution  of  meaning  will  depend  on  knowledge  the  viewer  already  has, 
knowledge  that  can  be  associated  with  the  incoming  information  (Heuvelman,  1987).  This  view  is  commonly  held  by 
many  researchers  and  recent  language  comprehension  studies  have  indicated  that  language  processing  involves  a context- 
dependent  knowledge  base  that  operates  in  an  integrative  and  elaborative  manner  (Anderson  & Ortony,  1975;  Barclay, 
1973;  Bransford,  Barclay,  & Franks,  1972;  Marschark  & Pavio,  1977).  Dillen  (1983),  Braden  & Walker  (1980)  and  Wise 
(1982)  also  point  to  prior  knowledge  of  the  individual  as  a variable  which  significantly  determines  how  an  image  will  be 
perceived.  Craik  and  Lockhart  place  prior  experience  as  a comparative  referent  within  the  memory  storage  system.  These 
researchers  go  on  to  describe  two  stages  of  the  memory  formation  process.  They  are: 

EARLY  STAGES...  the  analysis  of  physical  features  of  incoming  information... and 
LATER  STAGES...  concerned  with  matching  input  against  stored  knowledge  from 
past  learning,  and  with  abstracting  meaning." 

Some  researchers  have  consistently  found  that  when  images  are  utilized  which  include  people,  fixations  center  on  their 
faces  to  the  almost  total  exclusion  of  anything  else  in  the  image  (Buswell,  1935;  Chu  & Schramm,  1975;  Guba,  et  al, 
1964;  Yarbus,  1967).  The  implication  of  this  is  that  people  know  from  experience  that  faces  and  animate  objects,  in 
general,  are  primary  providers  of  contextual  information. 
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Animate  objects  receive  a higher  density  of  fixations  than  inanimate  objects  when  both  are  contained  in  the 
same  picture.  When  considering  portraits,  the  highest  density  of  fixations  occurs  on  the  eyes,  nose,  and  mouth  because 
these  areas  of  the  face  tend  to  convey  information  concerning  emotion  and  the  degree  of  physical  attractiveness  of  the 
individual.  (Yarbus,  1967  pg.  28) 

Structural  and  Contextual  Dominance 


A primary  concept  within  this  study,  in  relation  to  the  analysis  of  contextual  and  image  structure,  is  that  of 
dominance.  It  has  been  found  that  it  is  possible  to  define  an  image  in  terms  of  its  physical  structure  apart  from  its 
contextual  elements  (Friedman  & Poison,  1989;  Koroscik,  1984;  Marschalek,  1986).  Likewise  it  is  proposed  that 
linguistic  analysis  can  focus  on  both  the  structural  aspects  of  a sentence  apart  from  the  contextual  aspects  (Mason, 
Kniseley,  & Kendall,  1979;  Smith  & van  Kleeck,  1986).  In  each  of  these  four  categories  it  is  proposed  that  a dominance 
exists  which  may  be  utilized  in  comparing  these  categories,  resulting  in  a description  of  congruence,  non-congruence,  or 
ambiguity.  It  is  toward  the  definition  of  image  structure,  and  specifically  those  structural  elements  which  affect 
dominance,  that  this  study  is  directed. 

Since  no  models  emerge  from  the  literature,  we  may  turn  to  the  discipline  of  graphic  design  for  a description  of 
visual  form  by  Wucius  Wong  (Wong,  1972,  pg.  6)  which  seems  to  fit  a hierarchical  description  of  visual  form.  An 
adaptation  of  his  hierarchy  appears  below. 


Visual  Form: 

as  BASIC  ELEMENTS 
as  STRUCTURAL  ELEMENTS 
as  RELATIONAL  ELEMENTS 
as  CONTEXTUAL  ELEMENTS 


of  Point , Line , Plane,  Volume 
of  Shape,  Size,  Color,  Texture 
of  Contrast,  Direction,  Position,  Space,  Gravity 
of  Representation,  Function,  Meaning 


These  4 levels  of  visual  form  give  us  not  only  a language  to  describe  what  we  see  in  an  image,  but  also  a hierarchy 
which  is  harmonious  with  our  previously  stated  levels  of  analysis.  Describing  visual  form  in  this  manner  is  no  simple 
task,  for  in  order  to  compare  images  reliably  they  must  be  described  in  terms  of  their  attributes  at  each  of  these  four 
levels.  It  is  not  long  before  one  realizes  that  in  fact  it  indeed  takes  a thousand  words  to  describe  a picture.  This 
overwhelming  task  of  image  description  is  reduced  through  the  use  of  the  concept  of  dominance. 

In  discussing  the  structural  form  of  an  image,  the  most  appropriate  discipline  to  draw  from  is  art  criticism  or  art  history. 
Specialists  in  this  field  are  skilled  in  using  words  to  describe  images.  A common  practice  of  many  art  historians  is  to 
reduce  what  may  seem  to  be  a very  complex  image  to  a few  general  shapes,  colors  and  textures.  An  example  of  this  is  a 
description  of  Goya's  "The  3rd  of  May  1808:  The  Execution  of  the  Defenders  of  Madrid"  (Fig.  1) 
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Its  organic  structure,  based  on  triangles  and  strong  diagonals,  is  peculiarly  fitting  to  the  theme,  and  its 
neutral  colors  in  grays  and  browns,  with  a splash  of  red  in  the  pool  of  blood  heightens  its  emotional 
impact." (Gardner,  1959, pg.  443 ) 

One  item  to  note  from  this  description  is  the  implication  that  these  structural  elements  are  arranged  to  produce 
an  emotional  impact  and  not  a linguistic  one.  This  underscores  the  dual-coding  caveat  of  images  being  processed  in  the 
affective  domain  of  cognition.  This  description  is  very  brief  and  encapsulates  hundreds  of  individual  elements  into  gross 
generalizations.  It  is  necessary  to  exclude  lesser  elements  if  we  must  focus  on  the  dominant  ones.  It  is  interesting  to 
note  that  even  this  brief  description  alludes  to  contextual  elements  of  the  "theme"  and  the  "pool  of  blood"  and  is 
obviously  formed  as  a caption  intended  to  be  read  while  viewing  the  painting.  This  underscores  the  difficulty  in 
separating  contextual  elements  from  structural  ones. 

There  are  hundreds  of  individual  structural  elements  including  at  least  22  people,  16  faces,  all  of  their  wearing 
apparel,  weapons,  the  foreground  elements,  the  city  in  the  background  and  the  dark  night  sky.  That  we  can  attribute 
meaning  to  the  shapes  and  brush  strokes  alerts  us  to  the  fact  that  we  are  using  contextual  descriptions  of  structural 
elements.  If  we  use  entirely  contextual  descriptions  Goya's  painting  becomes  much  simpler.  We  could  describe  it  as  an 
image  consisting  of  three  groups  of  people  in  front  of  a wall  with  a city  in  the  background.  Another  description  of  the 
same  work  is  presented  to  illustrate  almost  an  entirely  contextual  description. 

"Here  the  blazing  color,  broad  fluid  brush  work,  and  dramatic  nocturnal  light  are  more  emphatically 
Neo-Baroque  than  ever.  The  picture  has  all  the  emotional  intensity  of  religious  art,  but  these  martyrs 
are  dying  for  Liberty,  not  the  kingdom  of  Heaven;  and  their  executioners  are  not  the  agents  of  Satan  but 
of  political  tyranny  — a formation  of  faceless  automatons,  impervious  to  their  victims'  despair  and 
defiance.  "(Janson,  1965,  pg.  479) 

It  would  seem  to  this  researcher  that  Goya  would  support  both  this  description  and  Gardner's  earlier  one  because 
he  chose  structural  elements  and  manipulated  them  in  such  a way  that  the  later  contextual  description  would  be  perceived. 
He  chose  to  manipulate  the  structural  elements  of  grays  and  browns  so  that  the  red  pool  of  blood  would  be  dominant. 

The  two  overlapping  triangular  shapes  point  like  arrows  toward  the  two  dominant  groups  of  people.  We  view  the  scene 
just  an  instant  before  the  pure,  bright  white  shirt  of  the  rebel  is  to  become  crimson  from  the  lines  of  the  rifles  pointing 
directly  at  it.  In  both  examples  stated  above,  the  structural  features  of  the  image  are  tied  directly  to  the  contextual 
features.  The  artists  have  selected  and  arranged  structural  elements  in  such  a masterful  way  that  the  desired  context  is 
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effectively  communicated  to  the  observer.  Another  way  to  state  this  is  that  there  is  congruence  between  the  structural 
dominance  of  the  image  and  the  contextual  dominance  that  was  the  intent  to  communicate. 

Based  on  the  findings  in  the  literature  and  translated  into  the  terms  of  dominance,  context,  and  congruency,  the 
following  hypothesis  were  adopted  for  this  study  and  which  the  methodology  was  designed  to  test. 

Hj.  Congruency  between  structurally  dominant  elements  and  contextually  dominant  elements  will  result  in 
greater  attention  to  these  elements  than  in  less-congruent  situations. 

H2.  As  the  complexity  of  an  image  increases,  the  structural  dominance  will  decrease. 

H3.  As  structural  dominance  decreases,  attention  will  be  directed  more  to  contextual  elements. 

Methodology 

Subjects 

A total  of  21  subjects  were  used  in  this  study  drawn  from  a graduate  class  in  research  design  at  Indiana 
University.  13  subjects  across  two  experimental  groups  received  events  with  the  same  sequence  of  images,  the  verbal 
context  being  the  only  difference  between  groups.  8 subjects  made-up  a third  control  group  exposed  to  images  only. 

Definition  of  Variables 

The  Contextual  Dominance  Variable 


he  verbal  contextual  dominance  variable  was  defined  as  a spoken  phrase  immediately  preceding  exposure  to  an 
image.  For  each  stimulus  event,  contextual  dominance  was  derived  from  the  nouns  within  each  phrase  and  then  given 
points  that  were  interpreted  in  terms  of  the  image.  Due  to  the  need  to  compare  the  verbal  contextual  variable  to  the 
image  structural  variable,  the  contextual  dominance  was  described  in  terms  of  the  referents  appearing  in  the  image  sectors. 

The  Image  Structural  Dominance  Variable 


The  image  was  described  in  terms  of  two  criteria,  structural  dominance  and  degree  of  complexity.  The  first  level 
of  image  description,  or  structural  dominance,  was  defined  in  terms  of  prioritized  sectors  of  the  image  as  assigned  by  a 
panel  of  visual  design  experts. 

The  Image  Complexity  Variable 

The  second  level  of  analysis  was  that  of  complexity.  Complexity  was  determined  simply  be  tabulating  the 
number  of  structural  elements  visible  in  the  image,  e.g.  10  lines,  20  circles,  6 shapes,  etc.  In  this  study  images  were 
grouped  into  simple,  medium,  and  complex  categories.  The  criteria  for  inclusion  in  the  simple  category  was  having  less 
than  10  structural  elements,  in  the  medium  category  by  having  more  than  10,  but  less  than  50  elements,  and  in  the 
complex  category  by  having  over  50  structural  elements  in  the  image. 

The  Congruency  Variable 


Identiiying  the  degree  of  congruency  between  two  groups  was  a statistical  process  of  shared  dominance 
comparisons.  Based  on  the  Experts'  rankings  of  structural  dominance,  phrases  were  constructed  which  either  referred  to 
this  dominance  or  referred  to  less  dominant  sectors  of  the  image.  Congruency  was  defined  as  an  independent  variable  with 
two  conditions.  A "Congruent  Condition"  existed  when  the  contextual  phrase  that  was  heard  referred  to  elements 
identified  as  part  of  the  image  structural  dominance.  The  phrase  which  referred  to  less  dominant  elements  within  the 
image  was  assigned  to  the  non-congruent  condition. 
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Materials 


Fifteen  images  were  divided  into  three  groups  of  five  according  to  the  above  complexity  criteria.  Structural 
dominance  was  determined  and  validated  by  a panel  of  visual  design  experts.  Two  contextual  phrases  were  written  for 
each  image  with  one  phrase  written  to  match  the  structural  dominance  of  the  image  and  the  second  phrase  focusing  on 
weaker  structural  elements  of  the  image. 

A sample  from  each  of  the  three  levels  of  complexity  follow: 

Simple  Complexity 


Fig  2 (Image  #5) 


Image  Structure  description 


Shapes:  2 circular  shapes  (1/5  width  diameter) 

Colors:  black  and  blue  on  white  background 

Textures:  solid  (no  textures) 

Relational:  diagonally  positioned  with  the  blue  circle  in  the 

upper/left  ninth  of  the  frame  and  the  black  circle 
in  the  lower/right  ninth  of  the  frame. 


Contextual  Phrases 


Congruent: 


Since  blue  knights  are  instinctively  afraid  of  black 
dragons,  he  ran. 


N on-Congruent: 


Since  black  knights  are  instinctively  afraid  of  blue 
dragons,  he  ran. 


Moderate  Complexity 


Fig  3 (Image  #8) 


Shapes:  1 hollow  triangle  shape  made  of  moderately  thick  lines. 

Colors:  black  on  white  background 

Textures:  solid  (no  textures) 

Relational:  Isosceles  triangle  fills  over  1/3  of  frame 


and  is  centered  in  frame. 
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Contextual  Phrases 


Congruent:  A pyramid  points  to  the  sky. 

N on-Congruent:  A pyramid  is  heaviest  at  its  base. 

Complex  Complexity 


Image  Structure  description 


Shapes: 

Colors: 

Textures: 

Relational: 


lines  and  a variety  of  shapes  but  mostly  rectangles, 

1/4  sphere,  circle  sections. 

grayish  beige,  bluish  green,  white,  yellow. 

rough  (stone-like),  and  slick 

symmetrical  3-dimensional  space, 

white  object  at  bottom,  illustration  at  top  of  image. 

Contextual  Phrases 


Congruent : The  center  alters  were  usually  simple 

with  only  two  candles  and  a cross. 

Non-Congruent:  The  biblical  phrase  ‘The  Lord  is  my  shepherd” 

is  reflected  in  the  decorations  around  the  half  dome. 
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Procedure 


Gathering  Eve-Movement  Data  Fie  5 


Subjects  were  asked  to  sit  in  a specialized  “head-stabilization”  chair  (Fig.  5)  A small  light  was  place  adjacent  to 

their  right  eye  and  a video  camera  was  trained  and  focused  on  the  eye  ball  and  a portion  of  the  light.  The 

strategy  was  to  record  the  reflection  of  the  light  on  the  cornea  of  the  subject  relative  to  the  stationary  light  bulb. 

Following  a registration  process,  the  stimulus  materials  were  presented. 

Recording  Eve-Movement  Data 

The  videotape  was  subsequently  played-back  one  frame  at  a time  and  the  motion  of  the  reflection  of  the  light  off 

of  the  subject’s  cornea  was  documented  s being  in  one  of  nine  sectors  of  the  frame. 

Statistical  Analysis 

The  Within-Group  analysis - 

Groups  A,  B,  C,  and  the  Expert  Group  were  included  in  the  analysis  for  each  of  the  15  images.  All  data  were 
converted  to  percentages  so  as  to  allow  for  a direct  comparison  between  the  stimulus  events  and  the  Expert  rankings.  A 
Kendall  Coefficient  of  Concordance  (W  ) was  computed  for  each  group  associated  with  each  image.  This  resulted  in  a 
rank-ordering  of  the  means  of  fixations  for  each  sector  of  the  image,  a Chi-Square,  and  a significance  value  for  each 
group.  If  the  result  of  the  Kendall  test  was  significant,  then  a more  rigorous  test  of  significance,  the  Friedman  Two-Way 
Analysis  of  Variance  by  Ranks,  was  calculated.  If  significance  was  acceptable,  calculated  by  the  Friedman  test,  then  a 
multiple  comparison  between  individual  rank-order  means  was  calculated  to  determine  the  significance  of  a sector’s 
dominance  (Siegal,  1988). 

Analysis  of  Eve-Movement  Data 

The  raw  data,  which  consisted  of  an  ASCII  file  of  75  data  pairs,  1,C  2,C  3,B  4,E  ...  75, H , were  read  into  a 
HyperCard  Stack  and  sorted  into  the  total  number  of  fixations  registered  in  each  of  the  nine  sectors. 

Determining  Dominance  from  Rank-Order  Comparisons 

Dominance  for  experimental  data  (that  which  was  derived  from  eye  movement  data),  as  well  as  the  ranking  by 
the  experts,  is  defined  as  those  sectors  which  produced  significance  from  the  Comparison  of  the  Means.  These  sectors  are 
then  said  to  represent  100%  for  the  Dominance  for  that  experimental  group  of  data  and  are  discussed  in  terms  of  the 
percents  which  each  sector  claimed  of  that  dominance.  For  instance,  in  the  example  bclow(Fig.  6),  the  dominance  for 
the  congruent  condition  of  image  8 (Sectors  B and  E)  is  89%  and  1 1%  respectively. 
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Figure  6 


Determining  Contextual  Dominance 

Contextual  dominance  was  determined  by  identifying  the  nouns  within  the  contextual  phrases  and  assigning  points  in 
relation  to: 

1.  their  position  within  the  sentence, 

2.  the  presence  of  visual  modifiers  that  were  also  in  the  image,  and 

3.  whether  they  were  a focus  of  attention  within  the  sentence. 

The  points  accumulated  for  each  noun  were  then  assigned  to  the  sectors  of  the  image  to  which  the  noun  referred.  The 
sectors  to  which  the  nouns  referred  represented  100%  of  the  contextual  dominance  and  were  represented  in  terms  of  the 
percentage  which  each  sector  claimed  of  the  dominance. 


The  Bctween-Group  Analysis 
Congruency 

A "Congruent  Group"  was  defined  experimentally  as  a group  which  received  an  audible  contextual  stimulus 
which  contained  a referent  to  a structurally  dominant  element  within  the  image.  The  assignment  of  an  experimental 
group  to  either  the  Congruent  data  group  (CG-S)  or  the  Non-Congruent  data  group  (NCG-S)  was  based  on  a strategic 
decision  by  the  experimenter  to  match  the  contextual  referents  heard  verbally  with  the  structurally  dominant  attributes 
within  the  image. 

The  Degree  of  Congruency  (°CG)  was  a post-hoc  statistic  used  to  describe  the  relationship  between  data  groups 
in  terms  of  the  overall  similarity  of  their  shaded  dominance.  This  statistic  produced  a number,  from  0 to  100.  with  the 
most  congruent  condition  being  100.  This  procedure  was  executed  as  follows  (see  Fig.  7): 

Corresponding  sectors  within  each  of  the  two  data  groups  being  compared  were  sequentially  added  together  producing  a 
"Sector  Sum".  Also  these  same  numbers  weie  subtracted  from  each  other  to  produce  a "Sector  Difference".  The  process 
was  repeated  for  each  sector  as  long  as  there  were  values  present  within  the  sectors  being  compared.  The  sector  sums  and 
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the  sector  differences  were  totaled,  then  the  difference  between  these  totals  was  found,  and  this  figure  was  divided  by  2 to 
produce  the  final  result. 

The  formula  for  determining  the  Degree  of  Congruency  (°CG)  was  as  follows: 

(Ksector  sum)  - Ksector  difference))  + 2 = °CG 


Figure  7 


% DOMINANCE  % DOMINANCE 


For  example,  comparing  the  EXP  group  with  the  CG-S  group  in  Figure  7: 

38+89=127  & 31+11=42 
^(Sector  Sum)=127+42=169 

89-38=51  & 31-11=20 
E(Sector  Difference)=5 1+20=7 1 

(169-71  )+2=  98+2=  49  = °CG 


Determination  of  Structural  & Contextual . Effects 

In  order  to  examine  the  effects  of  the  structural  dominance  and  contextual  dominance  on  the  Congruent  and  Non- 
Congruent  data  groups,  the  sector  of  primary  dominance  of  each  was  compared.  The  percentage  of  the  dominance  accrued 
in  the  sector  was  considered  to  be  representative  of  the  contextual  effect  for  that  group.  The  same  procedure  was  followed 
for  the  Non-Congruent  data  group. 

A similar  approach  was  utilized  to  determine  the  structural  effect.  The  sector  of  primary  dominance  for  the 
Expert  data  group  determined  which  sector  of  the  Congruent  or  Non-Congruent  data  group  represented  the  structural  effect 
for  each  group 

The  Effects  across  Complexity 

The  images  were  grouped  into  3 levels  of  complexity  — Simple  (images  1-5),  Moderately  Complex  (images  6- 
10),  and  Complex  (images  11-15).  By  plotting  the  contextual  effects  for  each  image,  an  examination  of  the  interaction 
effects  of  complexity  was  undertaken. 
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Comparison  of  Groups  by  Image 
Data  Group  Definition 


Results 


Since  treatments  for  congruency  were  randomized  among  experimental  groups  A & B,  and  later  recoded  into 
groups  of  congruency,  groups  A & B do  not  appear  as  units  in  the  results  section.  Instead,  each  stimulus  event  (the 
presentation  of  an  image,  with  or  without  a specific  phrase)  is  identified  as  being  either  congruent  or  non-congruent. 
These  events  are  coded  as  congruent  or  non-congruent  depending  on  whether  or  not  the  contextual  dominance  of  the  verbal 
phrase  matches  the  structural  dominance  as  defined  by  the  Experts.  This  recoding  establishes  6 groups  for  comparison. 


Experimentally  defined : 
(EXP) 

(FR-S) 

(CG-S) 

(NCG-S) 

Theoretically  defined: 

' (CG-X) 
(NCG-X) 


Ranking-Structural  Definition  / Expert  data  group 
Eye  Fixations  / Free-Viewing  data  group 
Eye  Fixations  / Congruent  data  group 
Eye  Fixations  / Non-Congruent  data  group 

Contextual  Definition^/  Congruent  Condition 
Contextual  Definition  / Non-Congruent  Condition 


Within  Image  Results 


Presented  here  are  the  results  of  one  image  from  each  of  two  complexity  groups.  Individual  within-group  data 
are  presented  graphically  via  a nine-sector  rectangle  containing  the  resulting  percentage  of  dominance  indicated  in  the 
appropriate  sector.  The  sector  which  is  primarily  dominant  is  outlined  with  a solid  box.  The  Kendall  W and  the 
Friedman  test  results  are  presented  adjacent  to  each  data  group’s  dominance  rectangle.  The  Degree  of  Congruency  is 
indicated  in  a box  joined  by  two  lines  connecting  the  groups  being  compared. 

After  the  presentation  of  the  results  of  each  of  2 images,  will  be  a discussion  of  the  results  across  the  15  images 
by  complexity.  This  comparison  will  utilize  only  the  Kendall  W and  the  primary  sector  that  relates  to  the  structural 
and/or  contextual  dominance. 


Image  #8 


Contextual 

Congruent 

A pyramid  points  to  the  sky. 

INI  INUAN1ILN1 
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Definition 

Non-Congruent 

A pyramid  is  heaviest  at  its  base. 

[N1  [N][AN][LN1 

1B,IE,1G,IH,U  3G,3H,3I 


Eye  Movement  Results 

Kendall  W.  Friedman: 

Significance,  Dominance,  & Degree  of  Congruency 
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Verbal  Summary  for  Image  #8 

Strategies  and  Anticipated  Results 


From  a visual  convention  standpoint,  a triangle  is  one  of  the  three  basic  graphic  shapes  (the  others  are  a circle 
and  a square).  This  shape  also  is  utilized  as  the  head  of  an  arrow  which  implies  direction  toward  the  tip  (which  would 
emphasize  sector  B if  structural  dominance  is  strong).  The  triangle  was  rendered  in  outline  form  so  as  to  highlight  the 
space  inside  the  shape  as  well  (sector  E).  The  contextual  cues  were  designed  to  emphasize  the  top  in  the  congruent 
condition  and  the  bottom  in  the  non-congruent  condition  in  the  anticipation  that  fixations  would  dominate  these  sectors  if 
there  was  a strong  contextual  dominance  effect. 

General  Stimulus.  Emit  Comwrmm. 

Ranking  of  sector  preference  was  significant  beyond  the  .01  level  for  all  data  groups.  Uniformity  of  image 
processing  was  moderate  for  the  Expert  data  group,  as  well  as  for  all  experimental  groups. 

•W (EXP)  =.5736  & FR-S)(CG-S)(NCG-S)  of  <.6884 

Coneruencv  Comparisons 

The  Degree  of  Congruency  (°CG)  between  what  the  Experts  identified  as  stiuctural  strengths  in  the  image  and 
how  the  Free-Viewing  data  group  viewed  the  image  was  moderately  strong,  °CG(EXP\FR-S)  =81  The  match  between 
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the  Free- Viewing  data  group  and  the  Congruent  data  group  was  moderate,  °CG( FR~S\CG-S)  =54,  and  the  match  with 
the  Non-Congruent  data  group  was  moderately  strong,  °CG(frs\NCG~S)  =72. 

Structural  & Contextual  Dominance  Comparisons 

A very  strong  Congruency  effect  was  evident  in  the  Congruency  data  group  with  this  image.  Nearly  90%  of  all 
fixations  were  accumulated  in  sector  B (D B(CG-S)  = 89%).  The  Structural  effect,  in  the  Non-Congruent  data  group, 
was  strong  enough  to  draw  enough  attention  from  the  contextually  dominant  sector  H producing  a strong  attention  to 
sector  B also  (Db(NCG-S)  = 43%).  There  was  some  contextual  effect  evident  in  the  Non-Congruent  data  group, 
evidenced  by  attention  to  sector  H (I>H(NCG-S)  = 29%),  compared  to  no  significantly  dominant  fixations  appearing  in 
sector  H for  the  Free- Viewing  data  group  (DH(FR-S)  - 0%). 


Image  #14 


Contextual  Definition 

Congruent 

The  center  alters  were  usually  simple  with  only  two  candles  and  a cross. 
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The  Lord  is  my  shepherd " is  reflected  in  the  decorations  around  the  half  dome. 
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Eye  Movement  Results 


Verbal  Summary  for  Image  #14 

Strategies  and  Anticipated  Results 

This  image  is  the  only  "realistic"  photograph  of  the  15  images.  The  picture  was  taken  from  the  poini-of-view 
of  someone  standing  in  the  center  of  the  room  at  an  eye-level  of  the  second  alter.  This  angle  accentuates  the  architectural 
perspective  lines  which  converge  just  under  the  center  window.  These  perspective  lines  alone  were  expected  to  draw 
strong  fixations  to  sector  E. 

The  first  ,,objectM  encountered  in  this  image  is  the  "floor  alter",  which  is  referred  to  in  the  congruent  condition's 
contextual  cue.  A focus  on  sector  H would  reinforce  a congruent  condition.  The  non-congruent  contextual  cue  refers  to 
the  mosaic  illustration  above  the  center  window,  which  is  the  largest  graphic  element  in  the  image,  as  well  as  the  largest 
architectural  element  (die  half-dome  in  sectors  A,B,  & C).  Due  to  the  complexity  of  this  image,  it  was  anticipated  that 
the  contextual  effects  would  be  strong  predictors  of  the  eye-fixations. 

General  Stimulus  Event  Comparisons 

Ranking  of  sector  preference  was  significant  beyond  the  .01  level  for  all  data  groups.  Uniformity  of  image 
processing  was  moderately  high  for  the  Non-Congruent  data, group,  moderate  for  the  Expert  and  Congruent  data  groups, 
and  low  for  the  Free-Viewing  data  group. 

•W r(NCG-S)  =.7590  &V/(EXP)(CG-S)  <-5718  &W(FR-S)  = 4633 
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Congruency  Comparisons 


The  Degree  of  Congruency  (°CG)  between  what  the  Experts  identified  as  structural  strengths  in  the  image  arid 
how  the  Free-Viewing  data  group  viewed  the  image  was  high,  °CG(£XP\FR-S)  =83.  The  match  between  the  Free- 
Viewing  data  group  and  the  Congruent  data  group  was  high,  °CG( FR-S\CG-S)  =82,  and  the  match  with  the  Non- 
Congruent  data  group  was  moderate,  °CG( FR-SINCG-S)  =69. 

Structural  & Contextual  Dominance  Comparisons 

A weak  congruency  effect  was  evident  in  the  Congruent  data  group  in  sector  H (Dh(CG-S)  = 38%).  A strong 
contextual  effect  was  evident  in  the  Non-Congruent  data  group  in  sector  B (PB(NCG-S)  = 40%).  A definite  focus  on 
the  center  of  the  image  was  evident  in  the  Free-Viewing  data  group  (J*E(FR-S)  = 43%).  Similar  to  the  preceding 
image,  if  in  fact  the  actual  structural  dominance  for  image  14  is  in  sector  E,  as  is  evidenced  by  the  Free-Viewing  data 
group,  then  both  the  Congruent  and  Non-Congruent  data  groups  demonstrate  contextual  effects  rather  than  congruency 
effects.  However,  it  may  also  be  said  that  both  groups  represent  partial  congruency  effects,  since  both  sectors  B and  H 
represent  secondary  and  tertiary  structurally  dominant  sectors. 

Between  Image  Results 

The  15  image  stimuli  presented  were  grouped  into  3 subgroups;  5 simple,  5 moderately  complex,  and  5 
complex  images.  The  Kendall  Coefficient  of  Concordance  measures  the  uniformity  of  the  subjects'  responses.  The 
Kendall  W (W  ) is  a rating  as  to  how  similar  each  subject's  response  was  to  other  responses  such  that  if  they  were 
identical,  the  W would  be  1.  Figure  8 illustrates  the  general  decrease  of  uniformity  of  the  experimental  groups  as  the 
complexity  of  the  image  increased. 

Uniformity  by  Complexity 

The  Experts  (EXP)  began  with  very  strong  agreement  (.9)  and,  as  the  complexity  of  the  image  increased,  their 
average  uniformity  dropped,  but  still  remained  in  the  strong  moderate  range  (.6).  The  Free-Viewing  Group  (FR-S),  on 
the  other  hand,  began  in  the  moderate  range  (.6)  and  finished  with  a low  uniformity  W (.42).  The  congruent  and  non- 
congruent  conditions  (CG-S  & NCG-S)  displayed  lessening  uniformity  as  complexity  increased,  but  stayed  in  the 
moderate  range. 

Comparisons  by  Primary  Dominance 

By  plotting  the  dominance  of  the  contextually  dominant  sector  of  both  the  Congruent  and  Non-Congruent  data 
groups,  a comparison  of  the  focus  of  these  groups  may  be  made  (Figure  8). 
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Figure  8 


Contextual  Focus  Across  Complexity 

by  Primary  Dominance  Sector 
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It  is  evident  in  this  graph  that  the  Congruent  data  group  consistently  exceeded  the  Non-Congruent  data  group's 
focus  on  the  contextually  important  element  of  the  image  for  the  simple  to  medium  complexity  images.  It  is  also 
apparent  that  with  images  12,  13, 14,  and  15  (within  the  Complex  category),  the  Non-Congruent  contextual  dominance 
exceeded  that  of  the  Congruent  data  group. 

Another  observation  evident  from  the  graph  is  the  strong  congruency  effect  for  the  moderately  complex  images. 

It  was  with  the  images  dealing  strongly  with  visual  conventions  that  the  Congruency  Effect  was  most  evident,  e.g.  that 
structural  dominance,  coupled  with  contextual  dominance,  had  the  greatest  effect.  Contextual  dominance  alone 
contributed  more  with  the  complex  images. 

Conclusions 

Response  to  the  Specific  Experimental  Hypotheses 

[Hl]The  first  experimental  hypothesis  posed  was  that  congruency  between  dominant  structural  elements  and 
dominant  contextual  elements  would  result  in  greater  attention  to  these  congruent  elements  than  in  non-congruent 
situations.  The  statement,  in  terms  of  this  experiment,  looks  specifically  at  the  contextual  dominance  generated  in  each 
of  the  Congruency  data  groups.  This  dominance  represents  not  only  a contextual  effect,  but  also  the  contribution  of  a 
structural  effect  as  well.  Because  of  the  combined  effect,  it  is  called  a congruency  effect.  In  the  Non-Congruent  data 
group  the  dominance  primarily  represents  the  contribution  of  a contextual  effect.  In  69%  of  the  cases,  a focus  on  the 
contextually  dominant  element  of  the  image  was  stronger  when  that  element  was  also  structurally  dominant 
A Congruency  Effect  was  particularly  evident  with  the  moderately  complex  images,  where  the  average  dominance  was 
34%  higher  than  that  in  the  Non-Congruent  condition.  These  results  lead  us  to  a conditional  acceptance  of  the  first 
hypothesis.  If  the  structural  dominance  is  strong,  then  congruency  between  structurally  dominant  elements  and 
contextually  dominant  elements  will  result  in  greater  attention  to  these  elements  than  in  less-congruent  situations. 

One  image  was  exemplary  in  demonstrating  Congruency  Effects  — image  #8.  Reviewing  the  structure  of  this  image  and 
context  will  clarify  the  types  of  "strengths"  referred  to  above.  The  word  "sky",  heard  in  one  group's  contextual  phrase 
referred  to  three  sectors.  A,  B,  & C,  which  were  clearly  related  to  the  upper  third  of  the  image.  The  other  group  heard  the 
word  "base",  which  refers  to  the  lower  third  of  the  image.  The  action  verb,  "points,"  which  was  heard  by  the  Congruent 
data  group,  coincides  with  the  visual  convention  of  an  arrow  (which  a pyramid  shares  in  shape)  that  reinforces  the 
congruency  between  contextual  cue  and  image  structure.  The  fact  that  the  object  was  an  isosceles  triangle  rather  than  an 
equilateral  one  increased  the  "pointing-up"  effect  even  more  (DB(CG-S)  =89%).  The  Non-Congruent  data  group  was 
directed  to  the  base  of  the  triangle  by  the  contextual  cue,  but  they  had  to  overcome  the  strong  image  structure  "pointing- 
up"  to  attend  to  the  base.  A very  small  portion  of  their  dominance  was  devoted  to  the  base  area  of  the  image  (DH(NCG- 
S)=29%).  This  image  produced  the  greatest  Congruency  Effect.  It  also  demonstrated  a strong  Structural  Effect  evident  in 
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the  inflation  of  sector  B in  the  Non-Congruent  data  group  (DB(NCG-S)=43%).  There  is  little  or  no  explanation  for  such 
dominant  fixations  to  occur  in  this  group  except  for  the  fact  that  the  image  structure  was  so  strong  as  to  draw  fixations 
into  sector  B. 

[H2]The  second  experimental  hypothesis  stated  that,  as  complexity  of  an  image  increases,  the  structural 
dominance  will  decrease.  Across  the  15  images  there  was  a continual  trend  downward  in  uniformity  of  agreement  of 
structural  dominance  by  the  Experts.  These  results  in  uniformity,  and  loss  of  agreement  on  structural  dominance, 
support  the  second  hypothesis.  Specifically  it  provides  evidence  that,  at  least  with  these  images,  as  elements  of  equal  or 
greater  structural  value  are  added  to  an  image,  the  structural  dominance  declines  in  strength. 

[H3]The  third  hypothesis  was  whether  or  not  structural  dominance  is  inversely  proportional  to  the  contextual 
effect;  or,  as  structural  dominance  decreases,  does  the  contextual  effect  have  a greater  impact  on  the  attention  patterns  of 
the  observer?  With  the  information  just  presented  on  uniformity,  if  the  hypothesis  is  true,  then  one  would  look  for  an 
increase  in  the  contextual  effect  from  the  more  complex  images  where  uniformity  was  low,  as  was  the  structural 
dominance. 

Referring  to  the  graph  in  Figure  8,  this  hypothesis  is  supported  by  the  Non-Congruent  data  group's  performance 
exceeding  that  of  the  Congruent  data  group  for  images  13  through  15.  As  was  mentioned  in  the  detailed  description  of 
the  complex  images,  the  structural  dominance  was  so  illusive  for  the  Experts  that  the  primary  dominance  they  indicated 
for  the  complex  image  set  never  matched  that  of  the  Free-Viewing  data  group.  This  success  rate  may  be  compared  to  a 
match  of  60%  in  the  moderately  complex  image  set  (images  6 through  10). 

A case  in  point  is  that  of  image  #14.  One  of  the  Experts  described  the  way  he  thought  that  persons  viewed  a 
photograph  was  spatially  (e.g.  the  same  way  one  would  if  he  were  to  be  walking  in  that  space).  In  other  words,  one 
would  look  at  the  first  thing  one  would  come  to,  then  the  second,  then  the  elements  surrounding  one's  immediate  space, 
then  beyond  (in  the  case  of  image  #14  this  would  produce  an  H,  E,  B progression  of  D ).  Another  Expert  suggested  that, 
since  everything  was  arranged  symbolically  in  a hierarchy  from  God,  to  Christ,  to  the  sheep,  and  finally  down  to 
common  man  at  the  altar  level,  that  the  viewing  patterns  would  coincide  with  that  order  (or  a B,  E,  H progression  of  D ). 
In  fact,  the  viewing  was  more  determined  by  the  context  presented  by  the  verbal  cue.  The  progression  of  D was  H,  E,  B, 
D for  the  group  which  received  a contextual  cue  relating  to  the  alters  at  the  bottom  and  center  of  the  image,  and  B,  E,  A, 

H for  the  group  which  heard  reference  to  the  illustration  in  the  half  dome.  This  supports  the  third  hypothesis  that,  as  the 
structural  dominance  decreases,  the  context  becomes  more  important  as  to  where  one  focuses  attention. 

This  finding  would  be  supportive  of  the  majority  of  research  to  date  in  the  area  of  complexity,  which  states  that 
simpler  visuals  are  more  predictable  and  memorable  (Loftus,  1972;  Luder  & Barber,  1984;  Friedman  & Liebelt,  1981 ; 
Palmer,  1975;  Heuvelman,  1987).  New  information  is  offered  from  this  experiment  regarding  the  relationship  of  Expert 
predictions,  Congruency  and  Free-Viewing  situations  in  terms  of  complexity.  These  variables  have  not  been  compared  in 
the  past. 

The  Concepts  of  Structural  and  Contextual  Dominance 

A major  contribution  of  this  study  is  its  identification  of  Structural  and  Contextual  dominance  as  factors  which 
define  Congruency,  accompanied  by  a methodology  for  measuring  them.  Many  studies  have  identified  the  structure  of  an 
image  as  being  a variable  (Heuvelman,  1987;  Marschalek,  1986;  Koroscik,  1984;  Nodine,  1982;  Molner,  1981  Nesbit, 
1978;  Dwyer,  1972),  but  none  have  defined  it  in  terms  of  dominance.  Many  of  these  same  researchers  have  presented  a 
verbal  or  written  phrase  as  a contextual  setting  to  provide  semantic  encoding  of  the  image,  but  none  have  compared 
contextual  and  structural  dominance  directly. 

To  the  visual  designer,  the  structural  aspects  are  of  utmost  importance,  since  these  are  the  elements  that  are 
being  manipulated.  The  sad  part  is  that  messages  are  designed  without  attention  to  the  structural  dominance,  or  visual 
design,  and  because  of  their  incongruity  with  the  message,  are  ineffective.  Research  is  done  without  concern  to  this  very 
strong  interacting  variable,  and  inconclusive  results  are  the  outcome.  The  sooner  that  structural  dominance  takes  its  place 
beside  contextual  dominance  as  an  equal  contributor  to  the  message,  the  sooner  we  will  begin  unraveling  many  of  our 
communication  issues  involving  images. 

* 

The  Concepts  of  Congruency  and  Complexity 

The  finding  in  this  study  that  Congruency  Effects  were  strongest  in  the  moderate  complexity  range  supports 
Heuvelman's  and  Koroscik’s  finding  that  congruency  can  be  higher  with  more  abstract  images.  This  appears  particularly 
true  when  familiar  visual  conventions  are  used  at  this  level  of  complexity.  This  study  only  partially  supports  other 
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findings  from  these  researchers  that  realistic  or  highly  complex  images  distract  to  such  a degree  that  congruency  cannot  be 
achieved.  This  study  supports  these  views  only  when  the  stimulus  consists  of  a weak  contextual  dominance  coupled 
with  a strong  structural  dominance.  In  these  situations  the  dominance  of  the  image  takes  over,  and  the  contextual  thrust 
is  lost.  However,  this  study  also  demonstrates  that,  especially  in  complex  images,  if  the  contextual  dominance  is  strong 
and  if  the  visual  referents  that  are  elements  of  the  weaker  smictural  dominance  are  also  to  by  found  within  the  image, 
then  a contextual  effect  will  also  be  strong,  because  of  the  contribution  of  a small  congruency  effect. 

Summary 

This  study  illustrates  that  when  the  structural  dominance  of  an  image  is  strong,  an  individual  viewing  this 
image  will  initially  attend  to  the  elements  contributing  to  structural  strength  rather  than  to  those  which  are  contextual. 
The  intent  of  most  educators,  in  the  presentation  of  an  image,  is  mostly  contextual.  If  a learner  who  rates  low  in 
knowledge  of,  and  motivation  toward,  the  specific  information  being  presented,  and  if  the  image  used  is  strongly 
structured  in  favor  of  a different  context  than  that  which  the  instructor  intends,  then  the  chances  are  high  that  the  learner 
will  not  process  the  image  in  a desired  manner.  If  this  lack  of  congruency  continues  during  a presentation,  confusion  on 
the  part  of  the  learner  will  be  the  most  likely  outcome.  The  converse  of  this  situation  is  the  desirable  state  of  affairs. 
Instructional  designers  must  gain  adequate  respect  for  the  structural  dominance  issues,  as  the  majority  already  have  for  the 
contextual  dominance  ones.  The  chances  that  learners  encountering  instructional  messages  will  find  the  pathway  to 
knowledge  must  greatly  be  enhanced.  Most  instructional  designers  do  not  have  the  skills  necessary  to  analyze  the 
structural  components  of  an  image  to  determine  the  structural  dominance.  Graphic  designers,  and  a newly  forming  role 
called  an  interface  designer  (mainly  involved  with  electronic  media  production),  do  have  these  skills,  but  most  often  are 
not  adequately  informed  in  the  subject  matter  or  instructional  strategies  to  determine  the  appropriate  contextual 
dominance.  A team  effort  is  necessary  to  assure  that  both  aspects  of  the  instructional  development  are  adequately 
pursued.  The  inherent  problem  is  that  one  person  talks  context  while  the  other  talks  structure,  and  no  progress  is  made. 
This  study  opens  the  door  for  communication  between  these  two  important  people  utilizing  the  rubric  of  Dominance.  If 
both  the  instructional  designer  and  the  interface  designer  respond  to  each  others  products  in  terms  of  the  dominance 
perceived,"  not  only  will  the  dialog  be  more  fruitful,  but  the  learner  (who  will  be  the  ultimate  "perceiver")  will  also  be 
brought  into  the  process. 

Placement  of  the  message  along  a quantitative  to  qualitative  continuum  becomes  a negotiation  item  for  the 
"team."  The  interface  designer  will  inherently  be  an  advocate  for  more  qualitative  messages  at  one  end  of  the  continuum, 
while  the  instructional  designer  most  likely  will  support  more  quantitative  solutions  at  the  other.  Villemain  (1966) 
supports  the  need  for  both  in  his  statement  that,  "Qualitative  mediations  are  instrumental  to  focally  cognitive  operations 
at  one  end  of  the  spectrum,  while  on  the  other  they  become  focal  with  cognitive  elements  assuming  the  instrumental 
role." 

Major  work  to  be  done  in  research  is  in  the  field  of  art,  not  to  quantify  it  and  make  it  rigid,  but  instead  to 
understand  it  adequately  enough  to  incorporate  it  into  the  message  design.  The  extent  that  anyone  creating  a message 
desires  a level  of  perfection  in  communication,  is  the  degree  to  which  one  embraces  a desire  to  deal  with  the  total 
continuum  from  information  to  art.  Let  John  Dewey's  challenge  be  our  motivation: 

As  long  as  art  is  the  beauty  parlor  of  civilization,  neither  art  nor  civilization  is  secure...  the  ideal 
human  community  is  dependent  upon  its  esthetic  component.  Rather  than  envisioning  art  as  an 
"efflorescence" , it  is  a condition  of  the  realization  of  democracy,  conceived  as  an  ideal  that  lends  a 
distinctive  character  to  all  aspects  of  life.  If  such  a view  is  tenable,  then  significant  revisions  in 
democratic  educational  and  social  theory  are  in  order.  (Dewey,  1934) 
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Introduction 


Given  the  present  emphasis  on  research  and  publishing  at  many  institutions  of  higher  education,  and  given  the 
dire  need  for  "socially  responsible"  research  that  can  contribute  to  effective  practice  in  many  types  of  learning 
environments,  and  given  the  contribution  res  rch  can  make  to  the  professional  vitality  of  instructional  technology 
faculty,  a panel  of  new  faculty  offer  here  the  challenges  they  have  faced  and  lessons  they  have  learned  in  attempting  to 
create  a research  agenda  for  themselves. 

To  provide  a point  of  reference,  a brief  biographical  sketch  will  be  provided  of  the  panel  members  and  the 
responder.  Then,  the  panelists’  comments  will  be  sorted  around  several  principle  topics  that  emerged  as  we  each 
individually  developed  our  panel  presentations.  Note  that  not  all  of  us'addressed  every  topic.  Finally,  the  responder's 
insights  and  recommendations  will  provide  closure  to  the  topic. 

Biographical  Sketches 

Dr.  Molly  Baker  is  a "practicing"  instructional  designer  at  Western  Illinois  University.  She  is  in  her  third  year 
(2nd  year  since  Ph.D.  completion)  serving  the  faculty  in  12  departments  who  are  learning  how  to  use  technology  in  their 
instruction  and  develop  hands-on  technology  experiences  for  their  students.  Through  the  Faculty  Instructional 
Development  Lab  which  she  runs,  faculty  are  encouraged  to  participate  in  workshops,  one-on-one  consultation  regarding 
course  design,  multimedia  development  teams,  individual  lessons  on  a host  of  technology-based  activities,  grant  writing 
for  technology  resources,  etc.  Her  research  agenda  is  driven  by  urgent  need-to-know  questions  and  practical  issues  of 
technology  utilization/integration. 

Drs.  John  Farquhar,  Jim  Quinn,  and  Steve  Harmon  are  assistant  professors  at  universities  who  adhere  to  the 
tenure-track  system  which  rewards  good  teaching,  an  active  research  program  leading  to  publication,  and  service  activities. 
They  are  in  their  3rd,  4th,  and  5th  years  respectively.  Dr.  Quinn  has  been  employed  at  a large,  research-oriented 
institution  and  the  Drs.  Farquhar  and  Harmon  have  been  working  at  institutions  where  teaching  and  research  are  more 
evenly  balanced.  Dr.  Reeves  is  a Senior  professor  at  the  University  of  Georgia,  is  widely  published  in  the  field  of 
Instructional  Technology,  and  has  focused  some  of  his  recent  writing  on  the  need  for  more  "socially  responsible"  research 
in  the  field. 


Perspectives  of  New  Faculty:  Building  a Research  Agenda 
Tip  #1:  Build  Skills  and  One  Research  Focus  Through  the  Dissertation 
Molly  Baker: 

When  I began  thinking  about  a research  question  for  my  dissertation,  I identified  three  criteria:  My  question 
must  suggest  a methodology  that  would  be  do-able  within  a reasonable  period  of  time;  it  must  be  interesting  to  me  (since 
I was  likely  to  spend  a large  chunk  of  time  on  the  task  and  build  several  years  of  follow-up  research  as  a new  faculty 
member  on  it,  as  well);  and  lastly,  it  must  be  valuable  to  professionals  other  than  myself-I  wanted  it  to  contribute  to  the 
field  in  some  meaningful  way.  It  took  months  to  find  such  a question  and  my  dissertation  chair  suggested  at  one  point 
that  I might  have  to  give  up  on  one  of  my  criteria! 

Now  I find  that  as  an  instructional  designer,  the  problem  is  not  finding  a researchable  question  at  all— I can 
identify  a couple  every  day  on  the  job!  The  problem  is  identifying  which  ones  to  pursue,  and  finding  the  time  and 
resources  to  do  it.  The  research  skills  I developed  in  graduate  school  have  been  essential,  but  the  research  focus  (effective 
teaching  on  interactive  television)  that  I anticipated  has  been  greatly  expanded  by  the  demands  of  my  position. 

John  Farquhar: 

In  retrospect,  my  research  as  a doctoral  student  should  have  positioned  me  to  perform  a scries  of  related,  follow- 
on  studies.  My  dissertation  involved  the  study  of  an  instructional  strategy  common  in  the  design  of  Intelligent  Tutoring 
Systems  (ITS).  At  the  time  of  my  defense,  I was  well  versed  in  the  ITS  literature  and  the  significance  of  my  work. 
Additionally,  I was  pleased  to  have  guided  the  study  so  that  additional  research  questions  were  naturally  to  follow.  Yet, 
despite  this  fortune,  I have  taken  a different,  less-traveled  path. 

During  the  final  phases  of  my  work  on  the  dissertation,  I became  a very  focused,  one-issue  researcher.  I 
intentionally  isolated  myself  physically  and  intellectually  from  the  Instructional  Technology  community  in  order  to 
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achieve  this  focus.  I found  my  isolation  to  be  a necessary  strategy  that  kept  me  on  track  to  giajuatc.  The  advantage  of  the 
strategy  was  immediately  evident  in  the  form  of  a successful  (i.e.  completed)  thesis.  The  disadvantage  of  the  strategy  did 
not  readily  appear  to  me  until  many  months  later. 

Steve  Harmon: 

It  all  started  in  the  winter  of  1986. 1 was  teaching  English  as  a second  language  in  a developing  country  and  was 
coming  to  the  end  of  my  two-year  contract.  I knew  that  I wanted  to  continue  to  work  in  some  way  with  developing 
nations,  and  needed  a graduate  degree  to  do  so,  but  I didn’t  want  to  do  the  same  old  development  stuff  that  every  body  else 
was  doing.  I stumbled  into  something  called  instructional  design  and  realized  I had  found  my  place. 

In  the  Fall  of  1987  I enrolled  in  the  Instructional  Technology  department  at  the  University  of  Georgia.  My 
intention  was  to  focus  on  Instructional  Systems  Design,  with  an  eye  toward  working  in  developing  nations,  but  along 
the  way  I got  sidetracked.  I became  interested  in  a new  technology  called  hypermedia.  Four  years  and  a couple  of  studies 
later,  I was  hired  as  a lecturer  in  Instructional  Technology  at  the  University  of  Houston-Clear  Lake.  I had  not  yet  finished 
my  dissertation,  but  had  collected  my  data  and  was  in  the  process  of  writing  it  all  up. 

Tip  #2:  Choose  An  Institution/Environment  That  Fits  Your  Interests 
Jim  Quinn: 

If  you  are  already  in  a faculty  position , you  already  should  know  what  the  balance  of  teaching,  research  and 
service  is  in  your  institution.  If  you  are  a graduate  student  (or  even  if  you  are  already  in  a faculty  position),  you  may  still 
be  deciding  what  balance  you  want  in  your  career.  For  example,  for  some  of  us,  we  are  in  positions  where  research  is  the 
primary  criterion  on  which  we  are  evaluated  for  promotion  and  tenure,  while  others  are  in  positions  where  teaching  is  the 
primary  criterion  on  which  we  are  evaluated.  However,  irrespective  of  the  type  of  institution  we  are  in,  we  will  all  have 
some  research/creative  activity  requirements.  Therefore,  my  first  suggestion  is  for  you  to  consider  what  would  be  the  best 
type  of  institution  for  you  to  work  in  given  your  interests  and  training.  If  you  wish  a considerable  amount  of  your  time 
to  be  involved  in  day-to-day  interaction  with  students,  teaching  classes  at  the  undergraduate  and  masters  level,  you  may 
w'sh  to  consider  an  institution  where  teaching  is  the  number  one  priority  and  research  is  the  number  two  priority. 
However,  you  should  be  aware  that  in  such  institutions,  you  are  going  to  have  to  be  involved  in  some  research/creative 
activity.  On  the  other  hand,  if  you  find  yourself  primarily  excited  by  being  involved  on  a continuous  basis  in  research 
projects,  supervising  Ph.D.  students,  etc.,  then  you  may  wish  to  seek  employment  in  a major  research  institution. 

Tip  #3:  At  First.  Watch  for  Research  Problems  and  Resources  as  You  Deal  With  the  Onslaught  of  Demands 
Tip  #4:  Then.  .Select  Research  Projects  From  These  Identified  Possibilities  and  Resources 
Jim  Quinn: 

It  is  impossible  to  be  perfect  in  teaching,  research  and  service,  and  sometimes  it  seems  to  me  to  be  impossible 
to  reach  the  required  level  of  performance  that  is  expected  of  all  new  faculty  in  each  of  these  three.  Therefore,  the  more  we 
can  do  to  integrate  each  of  these  the  better.  For  example,  in  my  case,  I am  very  interested  in  the  philosophy  of 
instructional  design  and  the  subsequent  implications  of  such  a philosophy  for  how  we  educate  instructional  designers.  I 
have  been  involved  in  attempting  to  integrate  theory  and  practice  in  instructional  design  by  having  students  in  my 
advanced  instructional  design  course  be  involved  in  real-life  projects  as  their  course  projects.  While  I am  aware  that  other 
faculty  in  many  institutions  have  been  doing  this  for  a long  time,  I was  not  aware  of  any  publication  on  this  issue,  so  I 
wrote  up  a report  on  this  for  ETRD  which  was  recently  published.  Since  then,  I have  done  one  more  research  study  on  a 
subsequent  iteration  of  this  course  in  a more  formal  research  manner  using  qualitative  research  techniques  and  I am 
currently  working  on  this  data.  I have  also  been  using  peer  evaluation  extensively  in  my  introductory  level  course  and 
have  collected  data  on  its  effectiveness  through  focus  group  interviews  and  am  preparing  this  for  publication.  I have  seen 
similar  good  empirical  work  being  done  on  the  education  of  instructional  designers  by  people  such  as  Peggy  Ertmer  and 
her  colleagues  at  Purdue  recently  on  the  use  of  case  studies  in  the  education  of  instructional  designers. 

Steve  Harmon: 

That  first  year  out  of  graduate  school  was  an  eye-opener.  I had  worked  hard  at  Georgia  and  was  reasonably 
competent  in  my  areas  of  interest.  I could  conduct  and  evaluate  research.  I could  design  and  teach  courses  on  a couple  of 
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topics.  I even  had  some  idea  about  writing  and  presenting  papers.  My  research  agenda  was  set;  finishing  my  dissertation 
was  my  primary  concern.  (Incidentally*  as  it  turns  out,  it  was  on  hypermedia.)  Much  to  my  surprise,  my  graduate 
preparation  had  not  covered  what  I ended  up  spending  most  of  my  time  on. 

I had  encountered  what  I now  call  “the  Janis  Phenomenon,”  after  the  Janis  Joplin  song  “Take  Another  Piece  of 
my  Heart.”  Far  from  being  able  to  work  happily  away  on  my  dissertation,  locked  safely  in  my  ivory  tower,  I was 
suddenly  besieged  by  a host  of  “real-world”  concerns  for  which  I was  totally  unprepared.  These  things  were  like  giant 
sponges  sucking  up  all  of  my  time.  I’d  like  to  spend  the  rest  of  this  discussion  going  over  those  concerns  and  how  they 
drove  my  research  agenda  for  the  next  few  years. 

The  first  thing  for  which  I was  unprepared  was  students.  I had  had  students  in  grad  school  of  course,  but  then 
they  were  mostly  safely  contained  in  classes.  The  students  I had  now  were  roaming  free.  They  wanted  me  to  advise  them. 
They  wanted  me  to  help  them.  And  some  of  them  wanted  me  to  tell  them  what  to  do  with  their  lives.  I quickly  learned 
why  several  of  my  professors  kept  a box  of  tissue  in  their  offices.  I had  become  some  kind  of  counselor.  I didn’t  have 
time  for  that. 

The  next  thing  I was  unprepared  for  was  colleagues.  In  graduate  school  I had  called  them  faculty,  and  mostly 
encountered  them  when  they  were  safely  contained  in  classes.  I only  had  to  deal  with  them  then  or  when  I chose  to  trap 
them  in  their  offices  and  force  them  to  give  me  advice.  But  now  these  people  had  offices  right  next  to  mine.  And  because 
I was  an  instructional  technologist  they  figured  I knew  something  about  computers.  I began  to  get  a constant  stream  of 
colleagues  coming  in  to  get  me  to  show  them  how  to  plug  in  a mouse.  And  who  knew  which  ones  would  ultimately  be 
on  my  promotion  and  tenure  committee.  I didn’t  have  time  for  that. 

The  third  major  thing  I was  unprepared  for  was  the  Dean.  I had  never  had  a dean  before.  In  graduate  school  the 
dean  was  someone  who  had  a big  office  on  the  first  floor  and  was  either  revered  or  reviled  depending  on  who  you  talked 
to.  The  only  time  I ever  saw  the  dean  in  graduate  school  was  once  when  I had  to  go  show  him  how  to  plug  in  his  mouse. 
Now  I had  this  Dean  who  actually  knew  who  I was  and  who  wanted  stuff  from  me.  I quickly  learned  that  the  translation 
of  the  word  “dean”  in  the  real  world  is  “boss.”  The  deal,  wanted  me  to  deal  with  my  program  (“increase  enrollment, 
specify  competencies,  tighten-up  evaluation  of  graduates”).  He  wanted  me  to  deal  with  the  school  (“increase  enrollment, 
bring  in  grants,  train  other  faculty”).  He  wanted  me  to  deal  with  the  university  (“give  me  an  example  of  the  great  things 
you  are  doing  to  show  the  president”).  He  wanted  me  to  do  a hundred  other  things  at  the  same  time.  And  of  course,  I 
didn't  have  time  for  that. 

I spent  the  first  few  years  trying  to  meet  all  of  these  demands  on  my  time.  My  research  agenda  was  dictated  by 
the  demands  of  the  Dean.  He  had  me  and  several  other  faculty  working  on  a major  grant  that  di- ' ’t  involve  hypermedia. 
What  time  I had  outside  of  that  was  quickly  taken  by  one  of  the  other  demands.  I seemed  to  have  a choice  of  either 
giving  up  research  in  hypermedia  or  giving  up  support  from  the  dean.  In  other  words,  I was  demanded  if  I did  and 
demanded  if  I didn’t. 

Over  time  I became  interested  in  what  the  Dean  was  having  me  work  on  (systemic  change).  I didn’t  lose  my 
interest  in  hypermedia,  but  instead  found  it  modified  to  fit  in  with  my  other  concerns.  More  importantly,  I began  to  see 
how  all  of  those  things  which  had  prevented  me  from  getting  a research  agenda  going  were  really  assets  which  could 
greatly  aid  my  research.  The  table  below  presents  the  time  sponges  I originally  saw  as  liabilities  and  how  I now  view 
them  as  assets. 
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Time  Sponge 

As  a Liability 

As  an  Asset 

Students 

Need  Advising 
Need  Extra  Help 
Need  Counseling 

Can  Conduct  Research 
Can  Challenge  You 

Faculty/Colleagues 

Want  Expertise 

Can  provide  expertise 
Can  provide  advice 

Classes 

Take  Time  for  Preparation, 
Delivery,  and  Evaluation 

Can  provide  Subject/ 
Environments  for  Research 
Can  be  test  ground  for 
disseminating  research 

Program 

Requires  time  and  effort  in 
recruitment  and  improvement 

Can  attract  better  students 
Can  enhance  reputation 
Can  provide  more 
colleagues 

School/College 

Requires  time  and  effort  to 
support  initiatives 

Can  provide  resources  for 
research 

Can  provide  environment 
for  research 

Committees 

Hold  seemingly  endless 
meetings  dealing  with 
seemingly  trivial  things 

Often  control  resources 
(money,  equipment,  space) 
needed  to  support  research 

The  Community 

Wants  expertise  and  free 
advice 

Can  provide  real  world 
problems  to  study 
Can  provide  resources 

The  Profession 

Requires  time  and  effort  for 
organizational  involvement, 
joumal/book  reviews,  etc. 

Can  provide  expertise, 
resources,  advice,  real 
problems,  reputation,  etc. 

In  short,  what  I found  was  that  while  external  factors  seem  to  drive  my  research  agenda,  I have  begun  to  learn  to 
steer.  Currently  I am  interested  in  creating  learning  and  performance  environments  on  both  a macro  and  a micro  scale. 
On  the  micro  scale  I want  to  build  multimedia/hypermedia  based  rich  learning  environments  for  individual  or  small 
groups  of  students.  On  the  macro  scale  I want  to  create  organizational  environments  in  which  these  micro  scale 
programs  can  be  implemented  successfully.  This  organizational  development  is  based  on  the  systemic  change  effort  that 
was  required  of  me  by  the  dean.  Rather  than  lament  all  of  the  conflicting  demands  on  my  time,  I now  seek  to  organize 
these  demands  so  that  they  can  be  met  while  at  the  same  time  supporting  my  own  research  agenda.  It  is  possible  to 
discover  a real  synergy  in  academia,  if  one  is  not  tom  apart  by  it  first. 


John  Farquhar: 

The  transition  from  doctoral  student  to  faculty  member  is  abrupt.  Within  a few  days  of  the  final  dissertation 
defense,  I began  my  career  as  a faculty  member,  preparing  for  new  courses  and  advising  students.  Even  with  a reduced 
teaching  load,  I spent  my  first  year  almost  exclusively  on  course  preparation. 

It  was  in  January  of  1995,  5 months  after  I started  my  faculty  career,  that  I received  a shock.  I had  just  become 
aware  of  the  ease  of  publication  on  the  World-Wide  Web.  My  recognition  of  the  potential  impact  of  this  new  technology 
hit  me  like  an  oncoming  train  from  behind.  I was  not  prepared  to  accept  that  many  new  and  exciting,  ’’cutting-edge” 
developments  had  completely  escaped  my  field  of  vision.  At  the  time,  I felt  antiquated  and  depressed. 

I now  recognize  that  the  World-Wide  Web  is  perhaps  the  fastest  growing  technology  in  all  of  history  and 
continues  to  hit  many  people  by  surprise.  Additionally,  I found  that  the  design  and  development  skills  I already  possessed 
were  easily  transferred  to  the  new  medium.  My  new  research  activities  are  now  largely  interested  in  the  instructional 
viability  of  networked,  computer-mediated  communication. 

At  this  point  in  my  career,  I am  still  trying  to  recover  from  the  isolation  I experienced  during  my  dissertation. 
Additionally,  I am  still  attempting  to  regain  the  research  momentum  that  I lost  when  I began  teaching  full-time.  Lessons 
to  be  learned  from  this  experience  are  (1)  isolation  from  the  field  for  even  short  periods  of  time  can  have  a significant 
impact,  and  (2)  the  momentum  of  dissertation  research  can  easily  be  lost  when  starting  a new  position. 
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Molly  Baker: 

Like  Steve,  I began  my  first  position  while  I was  still  working  on  my  dissertation.  The  demands  of  die  job  were 
constantly  compedng  for  the  time  I needed  to  complete  my  research/wridng.  As  I attempted  to  juggle  everything,  I began 
to  realize  that  although  my  primary  research  interest  would  likely  remain  effective  teaching  on  interactive  television  (the 
central  focus  of  my  dissertation  and  part  of  my  job,  as  well),  there  would  also  be  many  other  urgent  and  important 
avenues  to  pursue.  During  that  first  year  when  all  I could  manage  was  doing  my  job  and  getting  my  dissertation 
completed,  I started  a "research  ideas"  journal  which  included: 

potential  research  questions/topics  as  they  occurred  to  me  as  spin-off  projects  from  the  dissertation 
potential  research  questions/topics  as  they  occurred  to  me  through  working  on  real-world  problems  in  my  job 
potential  research  questions/topics  as  they  occurred  to  me  during  my  reading  in  the  literature  (both  professional 
and  more  "popular") 
potential  research  resources,  including: 

•potential  research  partners  on  the  faculty  at  WIU 
•potential  research  partners  elsewhere 
•bright  graduate  students 

•support  personnel  that  could  help  me  with  various  aspects  of  research 
•within-institution  grants  to  support  research  activities,  especially  in  the  summer 
•potential  publishing  outlets  (journals,  professional  magazines,  newsletters,  books,  etc.) 

•important  dates  that  might  impact  research  or  opportunities  to  share  results,  such  as  conferences, 
grant  deadlines,  on-campus  events,  etc. 

As  a result,  I have  lots  of  ideas  for  dissertation  spin-off  questions  I would  like  to  examine;  however,  the  real 
demands  of  my  job  as  the  primary  academic  support  person  for  faculty  teaching  courses  on  our  interactive  television 
system  at  WIU  has  helped  me  select  specific  projects  that  will  reap  immediate  rewards  in  terms  of  enlightening  future 
training  of  faculty  or  their  effective  practice  as  distance  teachers. 

Additional  research  activities  have  also  emerged  out  of  the  many  other  daily  demands  of  my  office.  Sometimes  I 
get  no  further  than  doing  a quick  Internet  search  or  literature  review  on  topics  such  as  consulting  practice,  multimedia 
development  teams,  innovation  adoption,  or  facility  design  for  teaching  with  technology.  Other  times,  I pursue  activities 
that  have  urgent  "need-to-know"  implications  for  future  planning.  For  example,  I may  conduct  extensive  needs 
assessments/surveys  or  in-depth  interviews  to  understand  the  immediate  needs  and  interests  of  "my"  faculty  so  that  I can 
better  plan  a range  of  services  and  workshops  for  them.  I may  plot  a formative  evaluation/journaling  activity  during  the 
development  phase  of  a multimedia  development  project  to  better  evaluate  my  role  and  that  of  my  graduate  students  in 
supporting  faculty  in  such  endeavors  in  the  future.  I regularly  do  a variety  of  evaluation  and  feedback  activities  following 
workshops  and  individual  lessons  I conduct,  with  the  specific  goal  of  improving  future  hands-on  faculty  development 
activities.  Additional  job  demands  that  prompt  research  activities  are  grant  evaluation  activities  and  requests  from  the 
Dean's  office  or  College  of  Education's  technology  committee  for  informed  guidance  on  planning  initiatives. 

Tip  #5:  Seek  Out  Others  to  Collaborate  With  on  Research  Projects 
Jim  Quinn: 

Do  not  try  to  be  a lone  ranger.  If  you  have  graduate  students  who  are  willing  to  work  with  you  on  research 
projects,  make  use  of  such  willingness.  They  will  get  experience  in  doing  research  and  you  will  get  help  in  running 
research  studies.  In  my  experience,  I have  found  it  to  be  of  great  value  to  have  two  to  three  research  students  working 
with  me  in  running  empirical  research  projects. 

Whether  or  not  you  have  graduate  students  available  to  work  with  you,  you  may  also  have  colleagues  who  are 
willing  to  work  with  you  on  such  projects.  Finally,  on  a broader  level,  keep  in  contact  with  other  new  faculty  that  you 
meet  at  conferences  such  as  AECT,  AERA,  etc.  - such  networking  offers  great  possibilities  for  the  kind  of  collaboration 
that  we  are  talking  about  here. 

On  a similar  note,  involve  people  who  have  skills  that  you  may  not  have.  In  my  case,  in  particular,  I am 
thinking  of  someone  who  has  skills  in  qualitative  research,  which  I am  not  expert  in.  I,  like  many  other  instructional 
designers  it  seems,  have  been  primarily  trained  in  quantitative  techniques  and  a significant  amount  of  the  research  I have 
been  involved  in  has  required  knowledge  of  various  qualitative  techniques,  and  I have  been  very  fortunate  in  working  with 
someone  who  is  knowledgeable  in  such  techniques. 
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Steve  Harmon: 

See  Table  in  section  on  Tip  #4  and  #5  for  suggestions. 

Molly  Baker: 

I have  found  it  a challenge  to  find  research  collaborators.  Our  master's  program,  for  example,  offers  a non-thesis 
option  for  our  students,  and  most  of  them  choose  it,  so  I virtually  have  no  graduate  student  assistance  for  research 
activities.  Additionally,  the  faculty  in  my  department  have  their  own  research  agendas  which  do  not  seem  to  overlap 
much  at  all  with  mine.  However,  I have  recently  identified  some  faculty  in  two  other  departments  with  interests  that  may 
blend  with  mine,  so  there  may  be  hope  for  future  collaboration.  Likewise,  I have  struck  up  a wonderful  relationship  with 
another  instructional  designer  that  I met  through  the  distance  education  listserv  (DEOS-L)  and  we  are  planning  on  doing 
some  joint  projects.  Also,  I have  a few  in  my  "research  journal"  that  I have  not  sought  out  yet. 

Tip  #6:  Develop  Discipline  for  Writing 

Molly  Baker: 

My  biggest  challenge  in  sharing  my  research  findings  with  the  wider  field  of  instructional  design  is  time.  As  the 
technology  activities  at  WIU  have  grown  and  my  specific  responsibilities  have  expanded,  I no  longer  have  blocks  of  time 
to  write.  I must  do  that  on  my  own  time  at  home,  which  for  a non-tenure  track  faculty  member  is  after  hours  or 
weekends.  I am  not  free  to  leave  for  part  of  a day  to  write  and  I have  a large  family  at  home.  I am  searching  for  a 
solution  to  tf  is  problem  this  year-including  more  self-discipline  on  my  part!  I have  a friend,  for  example,  who  writes 
every  day  from  7:30-10  and  does  not  open  his  office  door  until  then.  I do  conduct  many  presentations  at  conferences  and 
other  events  which  put  me  in  contact  with  other  ID  professionals  and  give  me  an  oral  opportunity  to  share,  at  least,  and  I 
plan  to  continue  that.  The  linear  process  of  research,  presentation,  paper,  and  publication  makes  sense  to  me.  I just  need 
to  find  a better  system  to  get  the  last  two  accomplished. 

Establishing  a Research  Agenda: 

Social  Responsibility,  Politics,  Goals,  Methods,  and  Practical  Steps 
Dr.  Thomas  Reeves 


Socially  Responsible  Research 

Although  many  mainstream  academic  researchers  will  disagree,  the  foremost  criteria  for  establishing  a research 
agenda  in  the  field  of  educational  technology  should  be  "social  responsibility."  Socially  responsible  research  "addresses 
problems  that  detract  from  the  quality  of  life  for  individuals  and  groups  in  society,  especially  those  problems  related  to 
learning  and  human  development"  (Reeves,  1995,  p.  2).  A more  traditional  view  of  the  purpose  of  research  is  captured  in 
this  statement  by  Fred  N.  Kerlinger  (1986),  author  of  one  of  the  best-selling  educational  research  textbooks: 

This  discussion  of  the  basic  aim  of  science  as  theory  may  seem  strange  to  the  student,  who  has  probably  been 
inculcated  with  the  notion  that  human  activities  have  to  pay  off  in  practical  ways.  If  we  said  that  the  aim  of  science  is 
the  betterment  of  mankind,  most  readers  would  quickly  read  the  words  and  accept  them.  But  the  basic  aim  of  science  is 
not  the  betterment  of  mankind.  It  is  theory,  (p.  9,  Kerlinger's  italics) 

However,  in  light  of  the  enormous  problems  faced  by  learners,  teachers,  and  trainers  around  the  world  as  well  as 
the  lack  of  impact  that  basic  research  has  had  on  educational  practice,  basic  research  in  an  applied  field  such  as  educational 
technology  is  a luxury  society  can  ill  afford.  (Of  course,  basic  research  in  related  fields  such  as  cognitive  psychology 
may  be  supported.)  A socially  responsible  view  of  the  purpose  of  educational  research  is  represented  by  this  statement  by 
Robert  Ebel,  a past  president  of  the  American  Educational  Research  Association  (quoted  in  Farley,  1982): 

....the  value  of  basic  research  in  education  is  severely  limited,  and  here  is  the  reason.  The  process  of  education 
is  not  a natural  phenomenon  of  the  kind  that  has  sometimes  rewarded  scientific  investigation.  It  is  not  one  of 
the  givens  in  our  universe.  It  is  man-made,  designed  to  serve  our  needs.  It  is  not  governed  by  any  natural  laws. 
It  is  not  in  need  of  research  to  find  out  how  it  works.  It  is  in  need  of  creative  invention  to  make  it  work  better, 
(p.  18,  Ebel's  italics). 
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Along  with  other  investigators,  educational  technology  researchers  are  usually  required  to  submit  their  research 
proposals  to  human  subject  review  committees  at  their  institutions.  Education  faculty  and  graduate  students  often 
complain  about  the  "onerous"  task  of  completing  this  "paperwork"  which  is  meant  to  ensure  that  research  activities  do 
not  harm  humans.  Despite  these  complaints,  I propose  that  universities  and  other  institutions  extend  these  reviews  to 
include  "human  benefits"  criteria.  Specifically,  if  educational  researchers  cannot  demonstrate  how  their  research  will 
benefit  human  learning  and  development,  it  should  not  be  approved  for  implementation  using  institutional  resources.  If  a 
human  benefits  review  was  taken  seriously,  I predict  that  much  of  the  pseudoscientific  research  that  characterizes 
education  would  be  exposed  and  eliminated.  (I  also  predict  that  such  a review  process  will  never  be  implemented  because 
of  the  political  investment  that  education  faculty  have  in  the  status  quo.) 

The  Politics  of  Educational  Research 

Honest  educational  researchers  in  colleges  of  education  will  admit  (albeit  not  as  publicly  as  Bracey,  1987,  quoted 
in  Sykes,  1988,  did)  that  the  primary  reason  they  conduct  research  is  so  that  they  will  have  something  to  publish  in 
research  journals,  and  thereby  attain  promotion  and  tenure.  Winning  the  "publish  or  perish"  game  is  the  ultimate  goal  of 
most  faculty,  and  the  game  itself  is  behind  the  rapid  proliferation  of  research  journals  and  conferences  in  recent  years. 

The  "peer  review”  process  that  should  guarantee  that  only  valid  research  gets  published  has  been  corrupted  by  the  sheer 
number  of  outlets  for  "scholarly"  publication.  In  turn,  the  review  processes  adopted  by  tenure  and  promotion  committees 
are  based  upon  quantitative  criteria  such  as  counting  the  number  of  refereed  journal  articles  an  applicant  has  listed  in 
his/her  vitae.  These  committees  rarely  have  any  members  who  could  evaluate  the  quality  of  these  research  studies  even  if 
they  had  the  time  and  motivation  to  do  so. 

Given  the  nature  of  the  game,  it  is  not  surprising  that  new  faculty  are  often  told:  "get  your  hands  on  some  data 
and  "publish  it  eveiywhere  you  can,"  as  I was  when  I was  a newly-minted  assistant  professor.  Virtually  no  one  directed 
me  toward  the  enormous  problems  that  confront  educators  and  trainers  in  every  comer  of  the  globe.  Ideally,  new  faculty 
and  graduate  students  would  be  mentored  into  socially  responsible  research  agendas  by  experienced  faculty,  but  the  sad 
truth  is  that  many  of  the  latter,  having  "won"  the  corrupt  tenure  and  promotion  game,  are  too  exhausted,  jaded,  and/or 
demoralized  to  continue  with  research  unless  it  leads  to  higher  accolades  or  substantial  pay  increases. 


The  Goals  of  Educational  Research 

So  what  is  the  educational  technology  faculty  member  or  graduate  student  to  do  to  establish  a socially 
responsible  research  agenda?  First  and  foremost,  you  must  be  clear  about  your  goals  beyond  the  mere  fact  of  academic 
survival.  In  short,  you  must  decide  on  the  type(s)  of  research  goals  to  which  you  can  ethically  devote  yourself.  Here  is  a 
partial  list  of  research  goals  appropriate  to  the  field  of  educational  technology. 

Theoretical:  A research  agenda  with  theoretical  goals  is  focused  on  explaining  the  phenomena  of  human  learning  and 
development  through  the  logical  analysis  and  synthesis  of  theories,  principles,  and  the  results  of  other  forms  of  research 
such  as  empirical  or  interpretivist  studies. 

Empirical:  A research  agenda  with  empirical  goals  is  focused  on  portraying  "how"  education  works  by  testing 
conclusions  related  to  theories  of  human  communication,  learning,  performance,  and  the  use  of  technology. 

Interpretivist:  A research  agenda  with  interpretivist  goals  is  focused  on  portraying  "how"  education  works  by  describing 
and  interpreting  phenomena  related  to  human  communication,  learning,  performance,  and  the  use  of  technology. 

Postmodern:  A research  agenda  with  postmodern  goals  is  focused  on  examining  the  assumptions  underlying  applications 
of  technology  in  human  communication,  learning,  and  performance  with  the  ultimate  goal  of  revealing  hidden  agendas 
and  empowering  disenfranchised  minorities.  The  postmodern  researcher  often  subscribes  to  a particular  political 
perspective,  e.g.,  feminist,  neomarxist,  or  multicultural. 
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Developmental:  A research  agenda  with  development  goals  is  focused  on  the  creation  and  improvement  of  innovative 
approaches  to  enhancing  human  communication,  learning,  and  performance  through  the  integration  of  technology, 
theory,  and  imagination. 

Action:  A research  agenda  with  action  goals  is  focused  on  evaluating  a 

particular  program,  product,  or  method,  usually  in  an  applied  setting,  for  the  purpose  of  describing  it,  improving  it,  or 
estimating  its  effectiveness  and  worth.  Action  research  often  directly  serves  the  needs  of  clients  in  education  and  training. 

Over  the  course  of  my  career,  most  of  my  own  research  has  been  driven  by  developmental  or  action  goals.  I 
have  found  this  to  be  personally  fulfilling,  and  in  the  final  analysis,  I believe  my  research  agenda  has  had  more  positive 
impact  on  real  needs  than  I would  have  had  if  I had  devoted  my  energies  to  theoretical  or  empirical  goals.  This  is  not  to 
say  that  my  research  agenda  has  not  been  shaped  by  the  theoretical,  empirical,  and  interpretivist  work  of  other  researchers, 
but  very  little  if  any  of  this  influence  has  come  from  within  our  field.  Unfortunately,  there  has  been  very  little 
interpretivist  research  in  educational  technology,  and  much  of  the  research  that  might  be  said  to  have  theoretical  or 
empirical  goals  amounts  to  pseudoscience  (Reeves,  1993). 

The  Methods  of  Educational  Research 

Once  you  have  decided  upon  your  goals  and  identified  a meaningful  research  problem  or  question,  you  are  ready 
to  choose  your  methods.  Choosing  research  methodologies  is  a complex  p.  ocess  that  has  as  much  to  do  with  your 
personal  epistemology  as  it  does  with  the  nature  of  your  problem  or  question.  Whether  you  are  an  empiricist,  a 
constructivist,  a critical  theorist,  or  a chaos  theorist  will  have  an  major  influence  on  your  preferred  methods  of  inquiry. 
Here  is  a partial  list  of  research  methods  used  in  our  field: 

Quantitative:  You  may  use  experimental,  quasi-experimental,  correlational,  and  other  methods  that  usually  involve  the 
collection  of  quantitative  data  and  its  analysis  using  inferential  statistics. 

Qualitative:  You  may  use  observation,  case-studies,  diaries,  interviews,  and  other  methods  that  usually  involve  the 
collection  of  qualitative  data  and  its  analysis  using  grounded  theory  or  other  ethnographic  approaches. 

Critical  Analysis:  You  may  choose  to  engage  in  the  critical  deconstruction  of  "texts"  and  the  technologies  that  deliver 
them  through  the  search  for  binary  oppositions,  hidden  agendas,  and  the  disenfranchisement  of  minorities. 

Literature  Review:  You  may  engage  in  various  forms  of  research  synthesis  that  usually  involve  the  analysis  and 
integration  of  other  forms  of  research,  through  methods  such  as  content  analysis,  frequency  counts  and  meta-analysis. 

Mixed  Methods:  You  may  develop  research  approaches  that  combine  two  or  more  methods,  usually  quantitative  and 
qualitative,  to  triangulate  findings  related  to  a particular  problem  or  question. 


How  do  you  get  started  with  a developmental  or  action  research  agenda? 

First  and  foremost,  you  must  identify  the  needs  in  your  environment.  Spend  a great  deal  of  time  in  other 
colleges,  K-12  schools,  and/or  business  and  industrial  training  centers.  Frankly,  in  an  applied  field  such  as  ours,  I think 
faculty  members  and  graduate  students  should  spend  at  least  two  days  a week  in  the  field.  After  spending  time  with  them, 
you  will  perceive  the  real  problems  faced  by  practitioners.  Make  their  challenges  your  challenges. 

Second,  collaborate  with  other  educators  and  trainers  in  preparing  research  and  development  proposals  to  find  the 
resources  needed  to  tackle  these  challenges.  Once  you  obtain  the  necessary  resources,  engage  yourself  and  your  students  in 
these  R & D projects,  and  carry  out  your  research  activities  within  the  context  of  meeting  these  challenges.  You  can 
even  extend  your  teaching  into  these  projects  by  engaging  students  in  assignments  that  are  situated  within  them. 

Third,  write  something  every  day.  whether  it  is  a memo  to  yourself  about  a new  idea,  an  e-mail  message 
detailing  your  progress  to  a colleague,  or  an  outline  for  a scholarly  paper.  Back-up  everything  electronically  and  set 
yourself  specific  goals  for  writing  scholarly  publications  for  periodicals  and  other  outlets.  Given  that  the  editorial  and 
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review  boards  of  many  refereed  journals  remain  mired  in  existing  paradigms,  look  for  opportunities  to  publish  your  work 
in  practitioner-oriented  magazines,  electronic  journals,  and  edited  books. 

Fourth,  carefully  document  your  scholarship  and  obtain  frequent  assessments  of  its  value  from  the  clients  with 
whom  you  work.  Although  tenure  and  promotion  committees  may  still  count  the  number  of  refereed  journal  articles  you 
have,  I believe  that  a well-documented  portfolio  of  scholarly  work  that  is  also  socially  responsible  will  not  be  ignored. 
Architects,  artists,  and  others  in  academe  have  developed  alternative  criteria  forjudging  scholarship  in  their  fields,  and  it 
is  time  for  educational  technology  to  do  likewise. 
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It  has  been  argued  that  organization  is  the  hallmark  of  good  instructional  materials  (Gagne',  1965).  Given  the 
emphasis  on  prose  and  written  materials  in  most  educational  systems,  the  sequencing  and  arrangement  of  subject 
materials  appears  to  influence  not  only  what  students  learn,  but  also  their  attitudes  towards  the  efficacy  and  importance  of 
instructional  materials.  Nowhere  is  this  more  important  than  in  the  preliminary  phases  of  the  design  and  development  of 
instructional  materials. 

Hartley  and  Davies  (1976)  state  that  in  addition  to  an  efficient  organization  of  instruction,  good  teaching  and  a 
well  organized  "bird's  eye  view"  of  a task  supplies  the  learner  with  a useful  perspective  of  what  will  subsequently  be 
encountered.  This  type  of  cognitive  organization  helps  the  learner  to  integrate  new  ideas  and  experiences  with  what  s/he 
already  knows. 

A review  of  the  results  of  previous  studies  obtained  shows  that  recall  and  conceptual  learning  of  prose  materials 
are  facilitated  by  the  use  of  advance  organizers,  behavioral  objectives,  orienting  questions,  pretest  and  posttest  questions, 
reviews,  and  lesson  prefaces  (Allen,  1970;  Ausubel,  1960;  Ausubel  & Youssef,  1963;  Barker  & Hapkiewicz,  1979; 

Duell,  1974;  Huck  & Long,  1973;  Peleg  & Moore,  1982;  Rothkopf  & Kaplan,  1972).  When  designing  and  developing 
instructional  materials,  a great  deal  of  emphasis  is  placed  on  instructional  strategies  that  serve  as  a basis  for  learner 
orientation. 

The  learning  models  by  Bloom  (1956)  and  by  Krathwohl,  Bloom  & Masia  (1964),  classify  instructional 
objectives  into  taxonomic  categories.  Gagne'  et  al.  (1988)  classified  criterion-based  human  performance  into  five 
categories:  verbal  information,  intellectual  skills,  cognitive  strategies,  motor  skills  and  attitudes.  In  order  to  optimize 
learner  performance  in  all  the  conditions  of  learning,  he  proposed  the  application  of  a nine-step  process  called  the  events 
of  instruction.  Events  1,2,  and  3-  gaining  attention,  informing  learners  of  the  learning  objective,  and  reminding  learners 
of  prerequisite  skills, -respectively  fall  within  the  rubric  of  preinstructional  activities.  The  preinstructional  activities 
investigated  in  this  study  were  advance  organizers,  performance  objectives,  and  structured  overviews. 

Advance  Organizers 

Studies  carried  out  during  the  past  30  years  present  a diverse  and  sometimes  conflicting  scenario  on  the 
facilitative  effects  of  advance  organizers.  Over  100  studies  have  investigated  the  effect  of  the  Ausubelian  organizer  at  one 
time  or  another  since  Ausubel’s  (1960)  preliminary  experiments  (Luiten,  Ames,  & Ackerson,  1980).  In  spite  of  the 
conflicting  results  on  the  efficacy  of  the  advance  organizer,  an  analysis  of  well-designed  studies  suggests  tentative  support 
for  the  organizer  technique,  especially  if  the  instructional  materials  are  not  well-organized  to  begin  with  and  the  target 
learners  have  limited  prior  knowledge  of  the  subject  matter. 

Variations  in  the  types  of  organizer  tasks,  recall  and  comprehension  measures,  validity  and  reliability  measures, 
and  student  characteristics  have  broadened  the  range  of  available  data.  This  has  resulted  in  methodological  and  theoretical 
implications  that  at  times  are  at  variance  with  Ausubel’s  advance  organizer  theory.  An  important  component  of  this 
study  was  to  narrow  the  definition  of  the  advance  organizer  in  order  to  make  it  easier  for  it  to  fce  operationalized. 

Performance  Objectives 

Research  on  performance  objectives  began  50  years  ago.  In  excess  of  100  studies  have  investigated  the  effect  of 
prescribing  behavioral  objectives  prior  to  instruction.  The  metamorphosis  of  performance  objectives  is  reflected  by  the 
variety  of  names  of  which  they  are  sometimes  known:  behavioral  objectives,  instructional  objectives,  mission 
objectives,  outcomes,  intents,  etc.  (Merrill,  1971,  Fig.  2,  p.  78). 

Duchastel  and  Merrill  (1973)  reviewed  several  studies  on  performance  objectives  and  came  up  with  a tentative 
generalization.  They  found  that  out  of  the  six  classes  in  Bloom's  (1956)  taxonomy  of  educational  objectives  (knowledge, 
comprehension,  application,  analysis,  synthesis,  and  evaluation),  the  utility  of  the  behavioral  objective  strategy  appeared 
to  be  more  useful  with  higher  levels  of  learning  tasks  which  call  for  analysis,  synthesis,  and  evaluation. 

Although  a number  of  studies  (e.g.,  Hartley  & Davies,  1976),  have  failed  to  support  the  hypothesis  that 
students  who  are  provided  with  performance  objectives  achieve  more  than  students  who  are  unaware  of  objectives,  a 
sufficient  number  of  investigations  have  confirmed  the  hypothesis  to  allow  a cautious  but  affirmative  opinion  on  the 
question  of  the  efficacy  of  behavioral  objectives.  The  conditions  under  which  behavioral  objectives  can  be  considered 
facilitative  remain  sketchy  and  cannot  be  generalized  across  global  populations.  An  important  component  of  this  study 
therefore  was  to  control  for  any  prevarication  in  the  manner  in  which  performance  objectives  are  defined  and 
operationalized. 


Structured  Overviews 

Generally  structured  overviews  have  been  grouped  into  several  categories  and  are  sometimes  referred  to  as  graphic 
organizers,  mapping,  webbing,  pyramiding,  graphic  summaries,  or  structured  overviews  (Cassidy,  1989).  For  purposes 
of  consistency  in  operationalization,  this  instructional  strategy  henceforth  will  be  referred  to  in  this  study  as  structured 
overviews. 

Researchers  have  found  that  using  structured  overviews  as  part  of  preinstructional  activities  can  facilitate  student 
learning  and  retention  of  instructional  materials  (Hartley  & Davies,  1976)  as  well  as  serve  as  a "blueprint"  for 
constructing  meaning  (cited  in  Greenwald,  1988).  In  studies  done  by  Carr,  Snouffer  and  Thistlewaite  (cited  in  Green wald, 
1988)  structured  overviews  have  helped  to  activate  prior  knowledge  crucial  to  understanding  passages  as  well  as  increased 
incidental  learning. 

Cassidy  (1989)  has  taken  a more  contemporary  view-one  more  closely  associated  with  the  context  in  which 
structured  overviews  are  used  in  this  study.  He  has  defined  a structured  overview  as  a cognitive  map  in  which  important 
aspects  of  a topic,  concept,  or  unit  of  study  are  identified  and  arranged  in  a visual  pattern  with  appropriate  verbal  labels. 
Again,  this  definition  of  structured  overviews  was  adopted  in  order  to  narrow  its  definition  as  well  as  make  it  fairly  easy 
to  operationalize. 

Hartley  and  Davies  (1976)  pointed  out  that  a great  deal  of  success  attributed  to  structured  overviews  is  thought  to 
lie  in  their  ability  to  emphasize  salient  points  as  well  as  to  select  and  condense  instructional  materials.  Overall, 
however,  the  results  of  the  studies  reviewed  showed  generally  facilitative  effects  of  structured  overviews  on  learning  and 
retention  of  textual  materials. 

Many  studies  (Duchastel  & Merrill,  1973;  Hartley  & Davies,  1976)  have  been  conducted  on  the  effectiveness  of 
these  variables,  although  very  few  of  them  have  been  documented  outside  the  confines  of  industrialized  countries.  In  fact, 
of  the  more  than  800  studies  accessed  from  the  ERIC  Documents  database  and  Dissertation  Abstracts  International  dealing 
with  the  subject  of  one  or  more  preinstructional  activities,  none  was  carried  out  in  Sub-Saharan  Africa. 

Despite  technological  advances  in  information  dissemination  throughout  the  world,  it  is  clear  that  printed 
materials  continue  to  form  the  bulk  of  instructional  resources  used  in  schools  and  training  environments  in  developing 
countries.  The  rationale  for  this  study  was  to  determine  the  efficacy  of  implementing  different  instructional  strategies 
using  printed  materials  and  subsequently  evaluating  their  capability  to  facilitate  learning.  This  study  was  carried  out  in 
Zimbabwe  where,  as  the  case  in  most  developing  countries,  the  principal  problems  appear  to  be  a rapidly  declining 
quality  in  educational  standards  and  a poor  instructional  design  infrastructure  (Moock  & Jamison,  1988).  Moock  and 
Jamison  (1988)  further  point  out  that  undesirably  low  levels  of  student  achievement  and  performance  experienced  in  sub- 
Saharan  African  countries  is  a direct  result  of  two  problems:  (a)  internal  inefficiency,  that  is,  the  educational  systems  in 
these  countries  make  poor  use  of  available  instructional  resources;  and  (b)  they  lack  the  resources  or  adequate  financing  to 
improve  the  present  curriculum  design  infrastructure.  In  Zimbabwe,  both  factors  are  operative.  The  country  also 
primarily  depends  on  printed  instructional  materials,  particularly  textbooks,  as  the  most  appropriate  media  for  improving 
the  quality  of  instruction. 

In  this  study,  the  use  of  selected  preinstructional  activities  was  investigated  in  Zimbabwe's  preservice  education 
corps.  Using  two  achievement  tests  with  different  performance  domains,  the  study  investigated  the  treatment  effects  of 
the  following  preinstructional  activities:  1 .)  advance  organizers;  2.)  performance  objectives;  and  3.)  structured 
overviews.  The  data  gathering  instruments  were  therefore  designed  to  measure  maximum  performance. 

The  specific  hypothesis  tested  in  this  study  was: 

There  would  be  significant  differences  between  the  population  means  of  posttest  scores  between  the  traditional 
approach  or  control  group  (CO)  and  each  of  the  preinstructional  strategies,  namely,  performance  objectives  (PO),  advance 
organizers  (AO),  and  structured  overviews  (SO).  The  rationale  behind  this  was  that  each  treatment  variable  was  expected 
to  enhance  recall  and  learning  of  specific  facts  and  concepts  as  they  pertainedto  the  lessons  by  both  focusing  the  learning 
effort  on  relevant  information  and  detracting  attention  from  incidental  material.  Through  the  process  of  orienting  the 
learners  to  what  they  would  have  to  know,  it  was  expected  that  they  would  be  better  prepared  in  learning  the  most 
important  and  relevant  parts  of  the  lessons  prior  to  taking  the  posttest. 

Participants 

A total  of  674  students  in  their  first  year  of  training  at  the  Gweru  (n=340)  and  Bulawayo-Hillside  Teacher's 
Training  Colleges  (n=334)  in  Zimbabwe  participated  in  this  study.  Of  these,  408  were  males,  193  females.  The  average 
age  of  the  participants  was  19  and  73  students  failed  to  specify  their  gender.  It  must  be  pointed  out  that  at  both 
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locations,  participant  mortality  in  the  second  reading  passage  was  considerable  due  to  the  fact  that  some  subjects 
conceded  familiarity  with  the  subject  matter. 

The  two  colleges  were  selected  because  of  the  fact  that  they  are  the  premier  preservice,  teacher  training 
institutions  in  Zimbabwe.  Bet  ween  them,  they  represent  approximately  60%  of  the  annual  pool  of  preservice  teachers 
who  are  trained  to  teach  in  Zimbabwe's  secondary  schools.  They  are  also  strategically  located  to  geographically  cover  the 
northern  as  well  as  southern  regions  which  divide  the  country's  two  indigenous  tribes,  the  Shona  and  Ndebele.  Through 
government  records,  the  two  institutions  were  predetermined  to  have  the  most  representative  student  body  in  the  country. 
Geographical,  tribal,  and  gender  representation  was  enhanced  by  a government  fiat  which  stipulates  an  admission  policy 
based  on  regional  quotas  for  both  colleges. 

All  participants  were  first  year  trainees  who  had  been  admitted  after  having  passed  Advanced  Level  secondary 
school  with  a first  class  or  high  level  second  class.  Their  classification  would  be  equivalent  to  college  freshman  in  the 
United  States  while  their  entry  level  grades  would  represent  A or  B-plus  respectively.  The  training  for  all  participants 
was  structured  to  be  inclusive  of  arts  and  science  subjects. 

To  increase  power  and  precision,  this  study  sought  the  participation  of  the  whole  student  population  from  both 
colleges.  As  a matter  of  national  pride  and  duty,  all  subjects  were  asked  to  volunteer  their  time  in  participating  in  the 
study  as  the  results  were  to  impact  on  the  designing  and  development  of  Zimbabwe's  post-independence  curricula.  In  all, 
participants  represented  approximately  80%  of  the  freshman  student  population  for  both  colleges. 

Experimental  Design 

The  independent  variables  in  this  study  were  defined  as  advance  organizers,  performance  objectives,  and  structured 
overviews.  These  have  been  defined  in  the  introductory  chapter  of  this  study. 

Four  levels  of  treatment  variables  were  used:  control  (CO),  structured  overview  (SO),  advance  oiganizer  (AO), 
and  performance  objectives  (PO).  The  three  treatment  variables  (structured  overview,  advance  organizer,  and  performance 
objectives)  wen'  studied  in  isolation  from  one  another.  The  outcome  of  each  variable  was  expected  to  reflect  its 
contribution  to  the  study. 

Student  participants  at  the  two  colleges  were  randomly  assigned  to  one  of  the  four  treatment  conditions  by 
randomly  ordering  each  of  the  self-contained  instructional  materials.  In  classes  that  averaged  roughly  100  participants  per 
session,  a random-ordering  system  that  involves  randomly  ordering  and  marking  each  of  the  four  types  of  instruction,  say 
1 , 2,  3,  and  4,  and  distributing  them  was  used. 

In  addition,  all  modules  representing  different  treatment  conditions  were  color  coded.  After  random  assignment 
of  the  treatment  conditions,  participants  were  assigned  to  four  sections  of  the  room  each  representing  their  module  by 
color.  Student  participants  at  each  location  were  asked  if  they  were  familiar  with  the  selected  reading  passages  before  they 
began  with  the  exercise.  Those  who  had  any  prior  knowledge  of  the  narratives  were  dropped  from  the  sample. 

Participants  were  also  asked  to  record  the  time  they  started  and  finished  both  the  reading  module  and  the  posttest.  Each 
student  was  given  45  minutes  to  read  a passage,  and  a fifteen-minute  posttest  followed  immediately.  This  process  was 
repeated  with  the  delivery  of  the  second  set  of  the  written  modules  and  posttest. 

The  achievement  measures  were  recall  and  conceptual  learning  as  determined  by  the  administration  of  two 
immediate  posttests.  A simple  one-way  analysis  of  variance  (ANOVA)  design  was  utilized  to  analyze  the  data. 

A technique  proposed  by  Glass  (1978;  by  which  treatment  effects  may  be  quantified,  standardized,  and  compared 
using  the  "effect  size"  statistic  (E.S.)  was  also  used.  The  rationale  behind  this  was  to  examine  all  effect  sizes,  regardless 
of  magnitude,  and  the  degree  to  which  they  were  worthy  in  their  own  right. 

Materials 

In  this  study,  two  reading  passages  unfamiliar  to  the  participants  were  selected  °nd  developed.  The  rationale 
behind  using  two  instruments  was  to  determine  whether  uniform  results  could  be  found  after  applying  both  instruments 
to  participants  within  the  same  population.  An  additional  reason  for  using  two  lessons  with  two  achievement  tests  was 
to  increase  the  power  and  precision  of  the  overall  instrument. 

The  content  of  both  instructional  materials  covered  technical  and  nontechnical  subjects  so  as  to  be  inclusive  of 
the  participants  academic  background,  which  was  science  and  humanities.  The  first  reading  passage,  amounting  to 
approximately  800  words,  was  adapted  from  a textbook  on  still  photography  (Grimm,  1985).  The  passage  had  technical 
language  that  dealt  with  the  basic  operational  functions  of  the  35  millimeter  single  lens  reflex  camera.  The  title  of  the 
passage  was  The  Basic  35  Millimeter  Camera. 
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Refinement  of  the  reading  passage  included  eliminating  any  parts  of  the  text  that  were  deemed  superfluous  and 
potentially  distorting  or  confusing  to  the  participants.  All  sectional  headings,  subtitles,  pull  quotes  or  any  other 
attention  getting  devices  were  removed  in  order  to  avoid  any  interactions  with  the  treatment  variables. 

The  readability  of  the  text  was  verified  using  a computer  program  to  ensure  it  was  at  the  Zimbabwe  Grade  7 
(U.S.  8th  grade)  level  and  thus,  presumably  below  the  participants  maximum  reading  level.  The  reading  materials  were 
timed  for  reading  length  to  ensure  that  they  would  be  of  appropriate  length  given  the  permissible  time  allocated  to  the 
researcher.  The  second  reading  passage,  amounting  to  approximately  750  words,  was  adapted  from  a book  featuring  timed 
readings  (Spargo  & Wiiliston,  1980).  The  selected  passage  dealt  with  the  topic  of  American  bald  eagles.  This  passage, 
entitled  The  Bald  Eagle,  was  less  technical  in  presentation,  substance  and  style  than  the  first  passage.  Materials  were 
developed  by  the  researcher  into  a reading  text  by  combining  information  from  subject  matter  experts  in  the  area  dealing 
with  the  American  bald  eagle. 

In  order  to  avoid  any  interactions  with  the  treatment  variables,  the  same  refinement  procedure  used  for  the  first 
reading  passage  was  applied  to  the  second  lesson.  The  reading  materials  were  also  timed  for  reading  length  and 
readability  level. 

Because  the  first  reading  passage  was  more  technical  and  somewhat  more  complex  than  the  second  one,  an  item 
analysis  on  the  post-tests  of  both  passages  was  conducted.  This  was  to  indicate  the  percentage  of  participants  who 
correctly  answered  each  item  on  both  test  components. 

The  text  materials  were  formatively  evaluated  to  determine  whether  indeed,  they  were  appropriate.  Editorial 
advice  was  sought  from  a group  of  instructional  design  experts,  two  English  Professors,  and  two  high  school  students 
This  process  was  designed  to  ascertain  the  logical  flow  of  ideas  contained  in  the  instruments. 

A second  formative  evaluation  for  both  reading  passages  was  carried  out  in  Zimbabwe  under  circumstances 
closely  resembling  the  actual  experimental  conditions  using  teachers  from  a teacher’s  college  in  Harare.  The  initial 
developmental  strategies  were  applied  to  conform  to  the  original  intention  of  the  study. 

For  each  reading  passage,  an  advance  organizer,  a set  of  performance  objectives,  and  a structured  overview  to  be 
used  as  treatment  variables  were  developed  with  the  help  of  the  same  panel  of  experts  who  assisted  in  developing  the 
reading  passages  and  test  items.  Each  intervention  was  formatively  evaluated  to  maintain  consistency  between  the  lesson 
content  and  ideas  from  the  treatment  variables.  Special  consideration  was  given  to  layout  so  as  not  to  confound  the  target 
learners.  All  the  treatment  variables  were  printed  in  such  a way  as  to  be  distinct  from  the  reading  passage. 

The  two  achievement  tests  accompanying  the  reading  passages  were  designed  to  measure  two  levels  of  skills:  a) 
recall  of  information,  and  b)  knowledge  of  concrete  concepts.  A list  of  items  was  drawn  from  each  reading  passage  to 
represent  the  desired  performance  domain.  Test  items  were  divided  into  performance  categories  of  verbal  information  and 
concrete  concepts,  the  latter  being  a subcategory  of  intellectual  skills  (Gagne'  et  al.,  1988). 

Validity  and  Reliability 

To  obtain  a performance  indicator  that  verbal  information  had  been  acquired,  participants  were  asked  to  state  in 
writing  different  pieces  of  information  related  to  the  passages.  As  a performance  indicator  that  concrete  concepts  had  been 
acquired,  participants  identified  objects  by  their  properties  or  classes,  and  animals  by  their  species  or  special 
characteristics. 

All  verbal  information  was  organized  and  presented  in  statements  which  were  comprised  mostly  of  facts  or 
declarative  information.  Concrete  concepts  were  presented  by  widely  varying  their  characteristics  so  that  each  individual 
participant  was  asked  to  identify  by  circling,  choosing  or  checking  from  a group.  Concepts  such  as  species  or  types  of 
mechanical  devices  were  identified  in  this  manner  as  well. 

A uniform  procedure  was  used  on  both  achievement  tests  to  establish  content-related  validity.  This  procedure 
involved  developing  the  tests  based  on  a set  of  performance  objectives  (Mager,  1962). 

In  order  for  student  performance  to  be  legitimately  represented  by  a single  score,  each  achievement  test  or  subtest 
had  to  measure  a single  learning  domain  pertinent  to  each  given  written  instruction.  Each  achievement  test  used  in  this 
study  was  checked  for  internal  consistency  using  the  Kuder-Richardson  or  KR-20  formula. 

Procedures 

The  strategy  for  administering  the  instruments  was  for  the  researcher  to  travel  to  the  different  teachers’  colleges 
at  which  he  administered  the  treatment  and  instruments  with  the  assistance  of  members  of  staff  at  the  two  colleges.  All 
participants  were  advised  of  the  relevance  of  the  study  and  its  implications  regarding  the  development  of  Zimbabwe. 
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They  were  also  advised  of  the  implications  of  the  study  from  the  point  of  view  of  an  instructional  design  and  materials 
development. 

Analysis  of  the  Data 

This  study  design  involved  administering  two  achievement  tests  to  respondents.  Participants  in  the  first 
achievement  test  exercise  were  comprised  of  470  first  year  preservice  teachers  from  Gweru  and  Bulawayo-Hillside 
Teacher's  colleges.  This  group  was  referred  to  as  the  35mm  SLR  group.  Participants  in  the  second  exercise  were 
comprised  of  204  first  year  preservice  teachers  from  the  same  teachers'  colleges.  This  group  was  referred  to  as  the  Bald 
Eagle  group.  The  dependent  measures  for  this  study  were  recall  and  conceptual  learning  as  determined  by  total  posttest 
scores. 

The  results  of  the  data  analysis  are  presented  in  the  following  order:  posttest  performance  on  the  two 
achievement  tests,  the  35mm  SLR  post-test  and  the  Bald  Eagle  post-test.  The  Kuder-Richardson  reliability  coefficient 
alpha  for  the  two  sets  of  achievement  tests  data  are  shown  in  Table  1. 

Table  1 

Posttest  Reliabilities:  Kuder-Richardson  (KR-201  Reliability  Coefficient  For  the  35mm  SLR  and  Bald  Eagle  PQSt-tfiSiS 


Test  Group 

N 

KR-20  Reliability 

35mm  SLR 

470 

.585 

Bald  Eagle 

204 

.627 

The  cell  means,  standard  deviations  and  relative  performance  of  the  four  groups  are  presented  in  Table  2. 


Table  2 

Means  and  Standard  Deviations  bv  Treatment  Group  of  the  35mm  SLR  Group 


Treatment 

Groups 

n 

Mean 

E.S. 

SD 

S.E.  ERROR 

CO 

109 

13.670 

- 

2.506 

.240 

AO 

119 

13.899 

.09 

2.506 

.230 

SO 

120 

14.150 

.19 

7.430 

.678 

PO 

122 

14.008 

.13 

2.710 

.245 

Total 

470 

13.938 

4.354 

.201 

Note:  Maximum  score  is  1 8 for  the  posttest. 
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A one-way  ANOVA  was  also  computed  for  die  first  posttest.  Based  on  the  performances  of  each  group,  the 
differences  between  groups  on  the  35mm  SLR  post-test  were  not  statistically  significant.  Since  there  was  no  significant 
univariate  effect  at  alpha  .05,  any  further  analysis  could  not  be  warranted. 

The  results  of  participants  in  the  Bald  Eagle  group  are  summarized  in  Table  3.  Table  3 also  shows  the  relative 
performances  of  the  treatment  groups,  including  the  basic  sample  statistics. 


Table  3 

Means  and  Standard  Deviations  by  Treatment  Group  of  the  Bold  Eagle  Group 


Treatment 


Groups 

n 

Mean 

SD 

E.S. 

Std.Error 

CO 

53 

13.627 

2.154 

- 

.302 

AO 

50 

13.660 

2.269 

.01 

.324 

SO 

52 

14.980 

2.661 

.51 

.376 

PO 

49 

15.208 

2.103 

.75 

.303 

Total 

204 

14.359 

2.405 

.171 

Note:  Maximum  score  is  1 8 for  the  posttest. 

A one-way  ANOVA  was  computed  on  the  Eagle  posttest.  Based  on  the  performances  of  each  group,  the 
differences  between  the  control  group  and  the  performance  objective  group  were  found  to  be  statistically  significant.  The 
ANOVA  also  detected  significant  differences  F(.05;  3,  200),  p<.05  between  the  control  group  and  the  structured  overview 
group.  There  was  no  significant  difference  between  the  control  and  the  advance  organizer  group. 

A major  problem  in  the  technique  used  to  analyze  the  data  so  far  is  that  it  does  not  take  into  account  any 
positive  treatment  effect  that  fails  to  reach  statistical  significance.  Hence  studies  showing  positive,  yet  statistically  non- 
significant effects  are  given  no  further  analytical  consideration.  Such  results  may  be  biased  against  favorable  yet  non- 
significant results. 

A technique  proposed  by  Glass  (1978)  by  which  treatment  effects  may  be  quantified,  standardized,  and  compared 
using  the  “effect  size"  statistic  (ES),  was  found  appropriate  for  this  study.  It  is  important  to  note  here  that  the  intention 
of  using  this  effect  size  approach  was  not  to  test  statistical  significance  of  the  effect  size,  but  rather  to  offer  an  alternative 
in  looking  at  group  differences.  It  is  also  equally  important  to  note  that  use  of  Glass’s  Effect  Size  technique  was  not  to 
suggest  any  negation  of  non-significant  outcomes  from  Analyses  of  Variance  conducted  in  this  study. 

The  effect  size  data  for  the  35mm  SLR  and  the  Bald  Eagle  groups  post-tests  are  shown  in  Tables  2 and  3 
respectively.  Effect  Size  data  appear  to  suggest  that  the  treatment  variables  used  in  this  study  have  a facilitative  effect  on 
learner  recall  and  conceptual  learning  in  varying  degrees  and  magnitude. 

In  looking  at  data  from  the  35mm  SLR  group,  the  effect  size  range  is  from  .09  to  .19  suggesting  a somewhat 
less  than  useful  treatment  effect  in  as  far  as  the  advance  organizer  is  concerned.  The  effect  of  the  advance  organizer  in  this 
instance  was  of  no  practical  importance.  In  looking  at  the  effects  of  the  performance  objectives  and  the  structured 
overviews  groups,  the  difference  between  these  two  groups  when  compared  with  the  advance  organizer  group,  while 
larger*  is  still  negligible. 
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Table  3 effect  size  indices  appear  to  present  a somewhat  different  composite  picture.  While  the  effect  of  the 
advance  organizer  remains  of  no  practical  importance,  the  performance  objectives  and  structured  overviews  groups  appear 
to  have  moderately  useful  to  sizable  effects  on  learner  recall  and  conceptual  understanding.  In  looking  at  both 
achievement  tests,  the  trends  in  the  data  seem  to  be  fairly  consistent.  Results  seem  to  indicate  that  the  advance  organizer 
was  the  least  effective  preinstructional  activity  when  compared  with  performance  objectives  and  the  structured  overview. 

Discussion 

In  general,  the  findings  in  this  study  appear  to  be  consistent  with  those  of  previous  research  (Duchastel  & 

Merrill,  1985;  Barnes  & Clawson,  1975;  Gagne' et  al„  1988).  These  results  seem  to  indicate  that  performance 
objectives  may  have  the  most  beneficial  effect. 

Despite  the  small  differences  in  the  means  between  groups  in  the  first  achievement  test,  the  trends  in  the  data, 
when  compared  with  the  second  achievement  test,  seem  to  be  fairly  consistent.  In  both  achievement  tests,  performance 
objectives  come  out  relatively  superior  to  any  of  the  other  treatment  variables.  Structured  overviews  appear  to  rank 
second,  followed  by  advanced  organizers. 

With  respect  to  the  overall  framework  of  the  study,  several  issues  appear  to  require  some  explanation.  There 
remains  the  question  of  why  the  differences  between  the  treatment  groups  in  the  35mm  SLR  group  were  not  statistically 
significant  while  significant  differences  were  detected  between  groups  in  the  Bald  Eagle  group. 

An  initial  explanation  may  be  found  in  the  differences  in  content  between  the  two  passages.  The  35mm  SLR 
reading  passage  was  relatively  more  technical  than  the  the  Bald  Eagle  passage.  In  conducting  an  item  analysis  on  both 
test  components,  item  difficulty,  as  indicated  by  the  percentage  of  students  who  correctly  answered  each  test  item,  was 
much  higher  in  the  the  35mm  SLR  group  passage  than  the  the  Bald  Eagle  group  passage.  The  average  item  difficulty 
index  for  the  35mm  SLR  group  was  .70,  while  the  item  difficulty  index  for  the  Bald  Eagle  group  averaged  .76. 

Another  observation  here  is,  by  design,  advance  organizers  are  fairly  complex  to  describe  and  interpret,  and  hence 
they  are  likely  to  be  inconsistent  in  terms  of  how  effective  they  may  be.  The  impact  of  the  advance  organizer  as  it  is 
designed  in  this  study  may  have  had  been  more  pronounced  on  higher  level  learning  than  simple  factual  recall.  Simply 
committing  facts  to  memory  may  be  a less  demanding  process  than  understanding  material  to  the  point  where  learners  are 
expected  to  answer  questions  that  draw  on  information  not  explicitly  contained  in  the  material  to  be  learned.  Since  the 
advance  organizer  is  supposed  to  facilitate  comprehension,  and  the  structuring  together  of  old  and  new  information,  it  is 
likely  to  have  more  impact  on  higher  level  learning  than  on  factual  recall  and  recognition. 

By  comparison,  the  performance  objectives  and  structured  overviews  leading  to  both  reading  passages  in  this 
study  provided  specific  information  derived  directly  from  the  content  of  each  lesson.  Both  of  these  preinstructional 
strategies  provided  precise  and  unambiguous  information  upon  which  the  learner  was  expected  to  focus. 

In  the  context  of  a developing  country  such  as  Zimbabwe,  data  from  the  two  achievement  tests  administered  to 
preservice  teachers  appear  to  support  the  use  of  preinstructional  activities  in  the  design  and  development  of  lessons. 
Central  to  this  discussion  is  the  fact  that  traditionally,  the  teacher  within  the  Zimbabwean  educational  context  has  always 
been  viewed  as  supreme  within  a pedagogical  environment.  Instructional  materials  such  as  textbooks,  media,  and  other 
support  resources,  are  secondary.  The  instructor's  use  of  any  o'  hese  support  systems  is  discretionary,  not  mandatory. 
Coming  at  a time  when  the  government  of  Zimbabwe  is  trying  to  remove  the  vestiges  of  colonialism  from  its  curricula, 
these  findings  may  be  somewhat  important. 

The  findings  of  this  research  also  seem  to  suggest  that  stated  preinstructional  activities  may  have  a global 
pedagogic  value.  Furthermore,  it  is  important  to  note  that  this  study  may  be  one  of  very  few  ever  done  outside  the 
United  States  and  Western  countries  in  general. 

While  overall  the  present  findings  provide  limited  support  for  the  types  of  preinstructional  strategies  investigated 
in  this  study,  it  is  important  to  note  that  these  results  do  not  depart  from  pervious  research  on  the  same  topic.  For 
example,  in  the  experimental  phase  of  this  study,  the  effect  of  performance  objectives  appears  to  be  consistent  with 
results  from  previous  studies  (Barnes  & Clawson,  1975;  Duchastel  & Merrill,  1973).  This  may  suggest  that  the 
effective  and  efficient  use  of  preinstructional  activities  traverses  cultures  and  international  Iwundaries. 

While  it  may  be  useful  to  determine  the  practical  significance  of  the  use  of  preinstructional  activities,  future 
studies  may  also  find  it  equally  important  to  consider  the  costs  of  using  such  innovations.  Evidence  of  effectiveness  may 
then  be  tied  to  guidelines  and/or  standards  in  ways  that  deal  more  realistically  with  the  present  emphasis  on  efficiency  and 
accountability  for  the  use  of  scarce  public  and  private  financial  resources.  Better  understanding  of  the  instructional 
systems  design  approach  should  be  encouraged,  as  it  is  a method  which  has  already  proven  useful  in  demonstrating 
effective  instruction. 
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ABSTRACT 


The  purpose  of  this  study  was  to  examine  the  effects  of  spatial  ability  level  and  window  presentation  style  of 
tiled  and  overlapped  computer  displays  on  the  achievement  of  dental  hygiene  students. 

All  forty-three  first-term  Dental  Hygiene  students  enrolled  full-time  at  a University  School  of  Dental  Medicine 
were  the  sample  who  participated  in  this  investigation.  Phase  one  of  this  project  consisted  of  administering  the 
Differential  Aptitude  Test:  Space  Relations  Subtest  to  assess  each  subject’s  spatial  ability.  Individuals'  scores  were 
examined  using  a frequency  distribution.  Participants  who  ranked  greater  than  the  median  were  classified  as  having  high 
spatial  ability.  Scores  less  than  the  median  were  categorized  as  the  low  spatial  ability  group. 

Two  HyperCard  Stacks  were  produced  targeting  the  topic  of  Ergonomics  in  Dentistry.  The  stacks  were  designed 
with  identical  content  and  differed  by  displaying  the  information  in  one  of  the  two  window  presentation  styles.  Subjects 
were  randomly  assigned  by  spatial  ability  level  to  one  of  two  treatment  groups:  tiled  or  overlapping  displays.  During 
phase  two,  each  subject  executed  the  corresponding  computer-based  tutorial  program  and  completed  an  identical  module 
test.  A 2 X 2 analysis  of  variance  (ANOVA)  was  used  to  analyze  the  achievement  scores.  A disordinal  interaction  in 
terms  of  achievement  between  the  two  factors  was  identified  with  significance  at  the  .01  level. 

BACKGROUND 

Despite  limited  theoretical  and  empirical  research  to  justify  their  usage,  windowing  environments  are  widely 
used  as  primary  computer  interfaces.  The  visual  display  mechanism  of  a window  environment  appears  to  capitalize  on 
the  spatial  organization  of  the  computer  screen.  Windows  subdivide  the  computer  screen  into  distinct  sections  each 
containing  different  bits  of  information.  The  information  contained  in  the  windows  can  be  viewed  simultaneously  using 
tiled  displays  or  in  segments  via  overlapping  techniques. 

"Windowing  systems,  according  to  their  proponents,  are  inherently  easy  to  use  and  automatically 
* conducive  to  user  productivity.  But,  while  it  is  clear  that  some  windowing  systems  offer  benefits  to 

some  users  under  some  conditions,  we  still  have  little  understanding  of  the  implementation,  user,  and 
task  parameter  that  lead  to  increased  productivity  and  satisfaction"  (Billingsley,  1988,  p.  413). 

Since  their  inception  in  the  early  1970‘s,  windows  have  been  assumed  to  be  useful  based  on  user  feedback  and 
personal  preference  indicators  (Miller  & McMillan,  1984).  To  date,  however,  little  research  has  been  conducted  on  the 
effects  of  the  use  of  the  various  windowing  techniques  (Jonassen,  1989,  Marshall,  1986,  Bly  & Rosenberg,  1986, 

Cohen,  1985,  and  Bury,  Davies,  & Darnell,  1985).  Therefore,  careful  examination  of  the  desktop  screen  design  may  be 
an  essential  component  for  window  utilization. 

Windows  are  commonly  employed  in  computer-based  educational  programs  to  display  information  using  the 
same  or  different  data,  to  access  peripheral  program,  and  to  provide  links  to  other  multimedia  workstations.  Two 
common  types  of  windows  are  defined  by  their  placement  on  the  screen.  The  first  window  configuration  is  termed 
"overlapped"  i.e.  one  window  obscuring  another.  Overlapped  or  stacked  windowing  displays  simulate  a realistic  desktop 
with  sheets  of  layered  paper.  In  an  overlapping  window  environment,  the  learner  is  only  permitted  to  view  a section  of 
the  window  because  the  windows  are  overlaying  one  another  producing  a stacked  effect.  An  overlapping  window  system 
requires  a learner  to  utilize  his/her  visual  skills  in  a three-dimensional  arena. 

The  second  window  display,  referred  to  as  "tiled"  i.e.  windows  not  obscured  by  one  another.  In  a tiled  display, 
one  can  view  each  window  on  the  display  screen  without  any  obstruction  from  each  of  the  other  windows  (Miller  & 
McMillan,  1984  and  Jonassen,  1989).  Tiled  window  screens  display  information  to  users  so  that  they  may  view  the 
windows  simultaneously.  In  a tiled  window  system  the  learner  draws  on  his  visual  abilities  by  interacting  with  the 
program  in  two-dimensional  space.  See  Figure  1 for  the  visual  representations  of  overlapping  and  tiled  window  systems. 
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Figure  1.  Window  Systems 

Limitations  in  human  memory  are  fundamentally  important  considerations  with  regard  to  cognitive  performance 
of  any  task  including  the  manipulation  of  a window  system.  Tiled  and  overlapped  window  configurations  serve  as 
external  memory  devices  to  the  users  internal  memory,  therefore  extending  the  participant’s  internal  memory  capability. 
Windows  tend  to  alleviate,  in  part,  the  memory  load  of  the  learner  by  serving  as  ’'markers"  of  unfinished  tasks  as  well  as 
enabling  individuals  to  view  more  than  one  data  set  at  a time.  (Billingsley,  1988  and  Card,  Pavel,  & Farrell,  1984).  A 
dilemma  may  occur  with  the  learner's  new  found  ’’cognitive  freedoms".  New  cognitive  strategies  and  encoding 
requirements  may  need  to  be  developed  by  the  learner  when  in  the  windowing  environment,  a condition  which  may 
already  be  cognitively  overloading  to  the  user  (Sullivan  & Stephen,  1985).  Researchers  have  attempted  to  compensate 
for  the  potential  Memory  load  dilemma  by  choosing  window  environments  that  depict  true  world  experiences  as  well  as 
by  providing  practice  to  the  user  within  the  windowing  display. 

Windows  appear  to  serve  as  tools  for  learners  which  makes  it  possible  for  them  to  hold  information  in  memory, 
organize,  and  retrieve  it.  Documented  research  does  not  clearly  explain  how  this  tool  functions  as  a memory  device. 
However,  researchers  do  suggest  that  this  storage  structure  for  visual  information  should  be  compatible  with  human 
perception,  retention,  comprehension,  and  retrieval  processes  (Reilly  & Roach,  1986).  A global  presentation  of  the 
information  is  afforded  by  the  tiled  displays.  Tiled  configurations  permit  the  user  to  divide  the  display  into  small  chunks 
of  information,  view  several  bits  of  information  simultaneously,  organize  it,  and  hold  the  data  in  working  memory.  A 
tiling  system  makes  it  easy  for  users  to  keep  track  of  all  open  windows  by  visually  inspecting  the  display..." 

(Billingsley,  1988,  p.  420). 

Overlapped  displays  require  active  participation  of  the  user's  ability  to  hold  information  in  the  working  memory 
from  one  display  to  another  and  spatially  relate  the  data.  In  this  case,  the  windows  serve  as  data  reminders  or  "markers”. 
The  user  must  be  able  to  infer  that  information  is  present  even  though  it  is  covered  by  another  window  (Norman, 

Weldon,  & Shneiderman  1986).  An  overlapping  system  makes  it  possible  to  'lose'  or  forget  about  windows  that  are 
covered  by  other  windows  (Bury  et  al.,  1985).  Learners  must  be  spatially  astute  when  using  the  overlapped  screen 
design. 

Jonassen  (1989)  has  identified  six  major  purposes  of  windows  including  operational  qualities,  navigational 
functions,  organizational/explicative  modalities,  explanatory/help  factors,  metaphorical  operations  and  exploratory 
devices.  To  date,  little  empirical  and  theoretical  research  has  been  conducted  on  the  effects  of  the  use  of  the  various 
windowing  techniques  (Jonassen,  1989,  Marshall,  1986,  Bly  & Rosenberg,  1986,  and  Bury  et  al.,  1985).  Therefore,  the 
need  for  experimental  research  in  this  area  is  necessary  for  the  improvement  of  current  and  future  computer-based 
instructional  packages. 

Miller  and  McMillan  (1984)  infer  that  tiling  and  overlapping  window  choices  are  "basically  a matter  of  personal 
preference"  rather  than  based  on  specific  cognitive  strategies  or  skills  (p.  98).  This  theory  has  been  challenged  in  the 
proposed  research  project.  It  was  postulated  that  performance  would  be  enhanced  when  selective  matching  occurs  between 
level  of  spatial  ability  and  windowing  style.  It  was  hypothesized  that  individuals  who  demonstrate  high  spatial  ability 
would  perform  better  when  using  overlapped  windowing  environments.  In  contrast,  low  spatial  ability  learners  were 
expected  to  have  better  educational  outcomes  when  using  tiled  windowing  displays. 

Norman  et  al.  (1986)  express  a similar  thought:  "When  there  is  a mismatch  between  the  user's  visual 
expectations  about  the  pattern  of  the  display  and  the  actual  operation  of  the  system,  it  is  hypothesized  that  performance 
will  be  impaired.  On  the  other  hand,  if  multiple  windows  and  coordinated  screens  are  used  in  a manner  that  supports  the 
user's  expectations,  they  will  become  powerful  tools  for  creating  new  work  environments  and  increasing  the  visual  scope 
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of  the  user"  (p.  231).  The  authors  have  suggested  that  the  learner’s  spatial  ability  must  be  considered  when  designing  and 
using  window  screen  designs  in  computer-based  instructional  programs. 

By  using  windows,  data  no  longer  is  solely  represented  in  two-dimensions  via  tiled  displays;  but  rather  three- 
dimensionality  is  introduced  using  overlapped  techniques.  These  technological  changes  require  that  learner  factors, 
especially  spatial  ability  level  should  be  considered  in  computer-based  program  design.  To  date,  documented  research 
regarding  the  effects  that  spatial  ability  may  have  on  performance  outcomes  has  not  been  adequately  addressed  when 
windowing  presentation  styles  are  developed  to  display  computer  program  data. 

Primary  mental  abilities  such  as  spatial  ability  are  ’’cognitive  skills  that  enable  individuals  to  learn,  think,  and 
reason”  as  stated  by  Thurstone  (1938)  cited  in  Jonassen  and  Grabowski  (1993,  p.  63).  Jonassen  and  Grabowski  (1993) 
add  to  this  definition  by  stating  that  spatial  qualities  include  the  use  of:  ’’...visual  skills,  spatial  manipulation,  similarity 
of  visuals,  and  imagining  how  visuals  might  appear  in  other  orientations”  (p.  64).  Bishop  (1980)  has  identified  two 
categorical  demarcations  for  spatial  ability  which  make  a distinction  between  ’’low-level  spatial  abilities,  which  require 
visualization  of  two-dimensional  configurations  but  no  mental  transformations  of  those  visual  images,  and  high-level 
spatial  abilities,  which  require  visualization  of  three-dimensional  configurations  and  the  mental  manipulation  of  these 
images”  (cited  in  Cohen,  1985,  p.  29).  The  definition  of  spatial  ability  in  this  study  was  drawn  from  Bishop's  second 
category  and  defines  spatial  ability  as  both  the  visualization  and  manipulation  of  objects  (windows)  in  space. 

Learners  possess  a wide  range  of  spatial  ability  levels.  Continuing  research  has  investigated  differences  in 
spatial  knowledge:  how  individuals  acquire,  manipulate,  encode,  and  retrieve  information  (Thomdyke,  1980).  Highly 
spatial  individuals  possess  the  aptitude  to  remain  unconfused  by  the  changing  orientation  in  which  the  spatial 
configuration  may  be  presented  (McGee,  1979).  Spatially  astute  subjects  tend  to  use  visualization  strategies  more 
effectively,  therefore,  exhibiting  better  visualization  abilities.  Educators  may  overlook  the  learner’s  spatial  ability  and  the 
differences  among  learners’  abilities  when  designing  instructional  materials.  Computer  program  developers  may  also  be 
ignoring  this  mental  ability  when  creating  computer-based  packages. 

Windows,  one  type  of  graphical  user  interface,  call  upon  user’s  spatial  ability  to  a high  degree.  Tiled  displays 
represent  data  in  two-dimensional  space  whereas  overlapping  environments  depict  information  in  three-dimensional  space. 
Billingsley  (1988)  comments  on  window  presentation  style  by  adding  that:  "This  attribute  specifies  the  possible  spatial 
relationships  between  windows  and,  to  some  degree,  the  types  of  operations  that  can  be  performed  on  them”  (p.  419). 

If  spatial  ability  levels  interact  with  the  screen  designs  and  memory  requirements  of  the  tiled  and  overlapping 
window  environments,  then  one  should  consider  the  spatial  ability  of  the  learner  when  selecting  appropriate  screen 
displays  for  computer-based  instiuctional  packages.  The  purpose  of  this  study  was  to  investigate  the  interaction  between 
spatial  ability  and  window  presentation  styles  (tiled,  overlapping). 


METHOD 

This  study  followed  the  format  of  Aptitude-Treatment-Interaction  research  in  that  the  aptitude  of  learner  spatial 
ability  was  contrasted  with  two  window  presentation  formats.  The  research  design  employed  in  this  study  is  depicted  in 
Figure  2. 


Figure  2.  Research  Design 

The  research  sample  was  comprised  of  all  forty-three,  first-term  Dental  Hygiene  students  who  were  enrolled  full- 
time at  a University  School  of  Dental  Medicine  Dental  Hygiene  Program.  All  subjects  were  volunteers  for  this  study  and 
represented  100%  participation  of  the  Dental  Hygiene  class.  Forty-one  subjects  (95.3%)  were  female  and  two  (4.7%) 
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were  male.  The  majority  of  the  subjects  were  18-25  years  of  age,  had  a variety  of  educational  backgrounds  and  computer 
proficiency  levels. 

The  Differential  Aptitude  Test:  Space  Relations  Subtest  was  administered  to  all  Dental  Hygiene  students. 
Students  were  dichotomized  into  high  and  low  spatial  ability  groups  using  the  median  score,  26.8,  as  the  Mcut  off*  point. 
Students  scoring  above  the  median  were  defined  as  high  spatial  ability  and  those  scoring  below  the  median  were 
categorized  as  low  spatial  ability.  Students  from  each  group  were  randomly  assigned  to  one  of  two  treatment  groups, 
tiled  or  overlapped,  using  stratified  random  assignment  procedures. 

Two  HyperCard  Stacks  were  produced  by  the  author  targeting  the  topic  of  Ergonomics  in  Dentistry.  The 
tutorial  stacks  were  designed  with  identical  content  and  only  differed  by  displaying  the  information  in  one  of  the  two 
presentation  styles,  tiled  or  overlapped  displays.  Information  was  presented  using  various  forms  such  as  text,  audio, 
digitized  voice  responses,  diagrams,  figures  and  realistic  digitized  images. 

At  the  completion  of  the  assigned  computer-based  treatment,  an  achievement  examination  consisting  of  31 
questions  assessing  subjects*  knowledge  of  the  content  presented  in  the  program  was  administered  to  all  subjects.  The 
completion  of  all  parts  of  this  study  took  each  subject  approximately  one  hour. 

RESULTS 

A 2 X 2 ANOVA  was  used  to  analyze  the  research  results.  Spatial  ability  and  window  presentation  styles  were 
the  two  factors. 

The  means  and  standard  deviations  of  the  achievement  scores  by  spatial  ability  and  type  of  window  presentation 
style  are  presented  in  Table  1.  As  can  been  seen  from  the  table,  the  size  of  the  four  research  groups  ranged  from  10  to  12 
subjects.  The  total  possible  score  that  could  have  been  attained  on  the  achievement  test  was  3 1 . 

The  Statistical  Package  for  the  Social  Sciences  (SPSS)  Version  5.0  was  used  to  analyze  the  data.  Table  2 
displays  the  ANOVA  summary  for  the  achievement  scores.  The  interaction  between  spatial  ability  level  and  window 
presentation  style  produced  an  F value  of  7.53  which  was  significant  at  the  .01  level.  The  resulting  disordinal 
interaction  is  plotted  in  Figure  3.  Since  a significant  interaction  was  obtained,  examination  of  main  effects  were  not 
appropriate.  The  plot  indicates  that  subjects  who  were  identified  as  having  high  spatial  ability  performed  significantly 
better  in  the  overlapped  window  environment.  Conversely,  low  spatial  ability  subjects  scored  significantly  higher  using 
the  tiled  displays. 


Table  1 

Means  and  Standard  Deviations  of  Achievement  Scores  by  Spatial  Ability  and  Type  of  Window  Presentation  Style 
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Table  2 


ANQVA  Summary  Table  for  Achievement  Scores 
Source 


Spatial  Ability  (A) 
Presentation  Style  (B) 
AXB 
Error 


1 

1 

1 

39 


MS 

1.11 

0.45 

38.37 

5.10 


0.22 

0.09 

7.53 


p value 

0.64 
0.77 
0.01 


Mean 

Achievement 

Score 


Tiled  Overlapped 

Presentation  Style 

Figure  3.  Interaction  of  Level  of  Spatial  Ability  and  Presentation  Style 


DISCUSSION 

The  tiled  and  overlapping  windowing  environments  appear  to  have  served  as  external  memory  devices  to  the 
user's  internal  memory  source,  thereby  extending  the  participant's  internal  memory  capability  as  predicted  by  Billingsley 
(1988)  and  Card  et  al.  (1984).  Windows  appear  to  alleviate,  in  part,  the  memory  load  of  the  learner  by  serving  as 
"markers"  of  unfinished  tasks  via  overlapping  displays  or  by  enabling  individuals  to  view  more  than  one  data  set  at  a 
time  through  tiled  displays. 

In  this  study,  subjects  who  demonstrated  higher  spatial  skills  obtained  greater  educational  outcomes  from  the 
organization  of  the  overlapped  window  formats.  This  may  be  in  part  due  to  the  cognitive  organization  ability  of  the 
highly  spatial  individuals  who  were  able  to  mentally  hold  and  relate  the  data.  The  overlapped  windowing  system 
permitted  the  highly  spatial  learners  to  "mark"  data  and  employ  their  three-dimensional  visualization  skills.  The 
overlapped  displays  required  users  to  hold  information  in  working  memory  from  one  display  to  another  and  to  spatially 
relate  the  material. 

Conversely,  tiled  displays  enabled  low  spatial  ability  subjects  to  perform  better  when  viewing  information  that 
was  presented  side  by  side  in  a two-dimensional  configuration  via  the  tiled  windowing  environment.  Tiled  configurations 
permitted  the  user  to  fully  view  several  bits  of  information  simultaneously  and  hold  the  data  in  the  internal  memory 
source.  Since  individuals  with  low  spatial  ability  have  limited  visualization  skills,  these  subjects  were  better  able  to 
remember  tiled  program  data  because  the  information  was  displayed  with  open  windows  permitting  full  visual  inspection 
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of  the  screen  displays.  The  low  spatial  ability  group  was  not  required  to  mentally  manipulate  and  relate  the  data,  as  in  a 
overlapped  windowing  system.  The  tiled  display  appears  to  better  complement  the  low  visualization  ability  of  the  low 
spatial  ability  learner  group.  When  window  formats  were  properly  matched  to  the  learner's  spatial  capability  more 
successful  educational  outcomes  where  obtained. 

It  is  interesting  to  note  that  students  exhibiting  high  spatial  abilities  performed  poorly  when  using  tiled 
windows.  The  reason  for  this  finding  is  not  clear.  However,  it  may  be  concluded  that  these  learners  prefer  to  impose 
their  own  spatial  schemata  on  the  information  presented  and  do  not  perform  well  when  an  incompatible,  program  defined 
schema  such  as  presented  in  the  tiled  treatment  is  imposed  on  them.  Further  research  is  clearly  needed  to  investigate  this 
issue. 

The  disordinal  interaction  found  in  this  study  strongly  suggests  that  the  factor  of  user  spatial  ability  has 
significant  implications  for  the  types  of  window  formats  which  will  be  most  beneficial.  Clearly,  those  students  with 
high  spatial  skills  did  profit  most  from  the  perceptually  more  complex  overlapped  environments,  while  those  with  low 
spatial  abilities  needed  the  more  global  presentation  of  information  afforded  by  tiled  windows.  The  cognitive  process, 
spatial  ability,  appears  to  be  one  of  the  factors  that  needs  to  be  considered  when  selecting  computer-based  instructional 
packages  that  exhibit  windowing  systems.  Compatibility  of  the  learner’s  spatial  ability  level  with  the  appropriate 
window  environment  has  been  shown  to  benefit  the  educational  outcomes.  Therefore,  additional  research  studies 
paralleling  and  extending  the  current  project  are  recommended. 
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The  Perseus  Project  is  an  undertaking  begun  in  the  Classics  Department  of  Harvard  University.  It  is  devoted  to  the 
creation  of  a massive  interactive  hypermedia  product,  called  “Perseus,”  on  ancient  Greek  culture.  According  to  Crane  and 
Mylonas  in  their  1988  article:  “The  Perseus  Project  is  developing  interactive,  computer-based  materials  on  Greek 
Civilization.  These  are  designed  to  support  learners,  instructors,  and  researchers  as  they  explore  this  complex  subject”  (p. 

25) .  “Perseus’  main  audience  consists  of  students  who  are  classics  or  archaeology  majors  or  who  are  taking  a course  in 
classical  civilization,  and  of  graduate  students  or  scholars  in  these  fields”  (Mylonas,  1992). 

Perseus  is  in  the  tradition  of  and  builds  on  the  ideas  and  earlier  hypertext  projects  of  such  visionaries  as  Vannevar 
Bush,  Douglas  Engelbart,  and  Theodor  Nelson.  The  huge  storage  requirements  of  Perseus  were  anticipated  by  Bush 
(1945)  in  his  seminal  article  “As  We  May  Think”  in  Atlantic  Monthly.  The  basic  ideas  of  hypertext  systems  were 
expounded  by  Engelbart  in  his  1963  article  entitled  “A  Conceptual  Framework  for  the  Augmentation  of  Man’s  Intellect.” 
His  NLS  and  Augment  hypertext  systems  emphasized  three  ideas:  “a  database  of  nonlinear  text,  view  filters  which 
selected  information  from  this  database,  and  views  which  structured  the  display  of  this  information  for  the  terminal” 
(Conklin,  1987,  p.  22).  Nelson’s  Xanadu  hypertext  system  hoped  to  incorporate  all  the  world’s  literary  efforts.  Nelson 
claimed  that  ‘The  evolving  corpus  is  continually  expandable  without  fundamental  change.  New  links  and  windows  can 
continuously  add  new  access  pathways  to  old  material”  (as  cited  in  Conklin,  1980,  p.  23). 

Unlike  many  of  its  predecessors,  the  Perseus  Project  is  serious  attempt  to  move  an  extremely  large  hypertext 
database  from  theory  into  practice.  In  contrast  to  Xanadu,  Perseus  concentrates  on  a single  subject  limited  by  geography 
and  time.  Brown  University’s  Intermedia  delivery  system,  which  is  an  outgrowth  of  Nelson’s  work,  was  considered  for 
Perseus,  but  rejected  since  it  could  only  run  on  expensive  workstations  which  cost  $7000  to  $10,000  each  (Hughes, 

1988).  Apple’s  HyperCard,  introduced  in  1987,  appeared  just  in  time  to  be  adopted  as  the  delivery  system  for  Perseus. 

Why  is  the  Perseus  Project  Important? 

If  the  goal  of  creating  a huge  database  on  ancient  Greek  culture  were  the  only  thing  riding  on  the  success  or  failure  of 
the  Perseus  Project,  then  the  work  on  this  esoteric  project  would  receive  coverage  in  journals  on  the  classics,  but  be  little 
more  than  a footnote  in  the  history  of  hypertext  and  hypermedia  software.  However,  the  potential  implications  of  Perseus 
extend  far  beyond  the  study  of  Greek  history.  The  literature  about  hypermedia  and  hypertext  includes  frequent  references  to 
the  Perseus  Project.  In  journal  articles,  it  is  commended  as  a model  for  hypermedia  development.  Hughes  called  the 
Perseus  Project  “today’s  most  ambitious,  intriguing,  and  promising  application  of  computer  technology  to  academic 
instruction  and  research  in  the  liberal  arts”  (1988,  p.  1).  In  her  article  about  evaluating  Perseus,  Neuman  said:  “The 
project  is  innovative  in  its  development  process,  its  technological  sophistication,  its  range  of  potential  applications,  and 
its  intended  outcomes”  (1991,  p.  239). 

Gregory  Crane,  the  Editor-in-Chief  of  the  Perseus  Project,  and  Elli  Mylonas,  Managing  Editor,  claimed  in  their  1988 
article  that  they  “are  committed  to  making  Perseus  useful  in  as  many  different  academic  environments  as  possible”  (p. 

26) .  Further,  they  expect  that  “a  working  model  of  electronic  publication  will  exert  influence  far  beyond  the  realm  of 
classical  studies”  (p.  26)  and  that  they  will  “provide  a blueprint  which  others  can  initially  and  subsequently  improve”  (p. 
26). 

In  his  article  “Redefining  the  Book:  Some  Preliminary  Problems”  (1988b),  Crane  speculated  that  the  Perseus 
working  model  might  substitute  for  or  replace  traditional  means  of  publication.  Crane  and  Mylonas  (1988)  predicted  that 
ohee  the  material  is  gathered,  Perseus  5.0  and  subsequent  editions  could  be  produced  at  a cost  no  greater  than  printed 
books.  The  Perseus  Project  was  committed  to  producing  the  core  data  and  software  at  a price  equivalent  to  the  cost  of  a 
regular  required  textbook  (approximately  $40  in  1988)  so  that  it  would  available  to  individual  students. 

What  Is  the  Perseus  Project? 

The  Perseus  Project  is  known  in  the  literature  as  an  interactive  hypermedia  product,  but  it  is  also  a name  used  for  the 
entire  enterprise  which  created  the  product  and  continues  to  support  it  and  plan  for  its  future.  The  developers  tend  to  call 
the  enterprise  “The  Perseus  Project”  and  the  hypermedia  product  itself  “Perseus.”  This  paper  will  follow  the  same 
convention. 

Contents 

[Perseus]  will  include  almost  the  entire  surviving  body  of  Greek  Tragedy,  comedy  and  epic;  works  of  major 
historians  such  as  Herodotus  and  Thucydides;  and  substantial  portions  of  the  massive  surviving  works  of  Plato  and 
Aristotle.  There  will  be  color  images  and  measured  drawings  of  museum  objects  (such  as  sculpture  and  Greek  vase 
paintings),  plans  and  pictures  of  buildings  and  sites  in  Greece,  and  an  atlas  based  on  Landsat  images.  Much  of  the 
material  commonly  studied  in  courses  on  classical  Greece  will  be  included  in  the  database.  (Crane,  1988a,  p.  51) 
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Figure  1.  The  Perseus  Gateway 


Funding 

The  first  funding  proposal  for  Perseus  was  made  to  the  Annenberg/CPB  Project  in  January  1986  by  G.  Crane,  R. 
Grant,  V.J.  Harward,  A.  Henrichs,  T.  Martin,  G.  Nash,  and  D.N.  Smith.  Hughes  (1988)  claims  that  the  funding  proposal 
to  the  Annenberg/CPB  Project,  Apple  Computer,  and  Harvard  University  dated  May  24,  1988  which  covered  funding 
from  January  1989  to  December  1992  is  for  $3,390,541.  “When  completed,  Perseus  will  have  cost  between  $4  and  $5 
million”  (Hughes,  1988,  p.  3).  Crane  (1990)  confirmed  a figure  of  “approximately  $3  million  between  1987  and  1993” 
(p.  35).  Crane  and  Mylonas  (1988)  hoped  that  Perseus  would  “become  an  ongoing  concern  and  can  retain  its  vitality  long 
after  major  external  funding  has  ceased”  (p.  26). 

Development 

The  content  of  Perseus  was  the  work  of  a team  of  classicists  and  other  specialists  at  Harvard  and  Boston  Universities, 
and  Bowdoin,  Pomona,  and  St.  Olaf  Colleges.  Each  classicist  was  responsible  for  developing  one  or  more  of  the  topic 
areas  (Crane  and  Mylonas,  1988).  According  to  A.M.  Lewis,  a Perseus  Project  staff  member  at  Harvard  University,  ‘The 
programming  was  done  by  several  people,  including  Sebastian  Heath,  Neel  Smith,  and  our  current  programmer,  Jake 
Sisk”  (e-mail  communication,  November  18,  1993).  The  instructional  design  process  included  “our  Editor-in-Chief,  Greg 
Crane,  the  programmers  mentioned  above,  and  many  other  people”  (A.M.  Lewis,  e-mail  communication,  November  18, 
1993).  Kate  Withey  of  Willow  Design  has  been  given  credit  for  the  “user  interface  to  Perseus”  (Crane,  1992a).  Lewis 
gives  credit  for  the  “general  shape”  and  “very  existence”  of  Perseus  to  Crane.  In  fact,  Lewis  claimed  that  Crane  chose  the 
Perseus  name  when  “Odysseus”  was  ruled  out  for  having  too  many  syllables  and  starting  with  a vowel  (e-mail 
communication,  November  22,  1993).  There  was  no  instructional  designer. 

Delivery  System 

Perseus  1.0  will  run  on  a Macintosh  LC  with  a hard  drive.  Either  a CD-ROM  drive  is  needed  or  a file  server  on  a 
network  which  is  able  to  access  the  CD-ROM  on  which  Perseus  is  stored.  A videodisc  player  and  monitor  is  needed  if  the 
videodisc  is  to  be  used.  The  videodisc  has  full  motion  and  still  frame  video  with  “short  introductions  to  three  major 
sanctuaries  of  ancient  Greece:  Delphi,  Eieusis,  and  Olympia  These  sequences  can  be  played  as  continuous  video,  with  a 
voice-over  commentary  on  the  history  and  archaeology  of  the  sites”  (Crane,  1992b).  Still  images  in  Perseus  appear  on 
both  the  CD-ROM  and  the  videodisc. 
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Perseus  2.0  will  be  published  in  two  editions,  the  Comprehensive  Edition  on  a four  CD-ROM  set  and  the  Concise 
Edition  on  a single  CD-ROM. 


Future  Releases 

The  plans  in  1988  called  for  cumulative  releases  of  Perseus  2.0,  3.0,  and  4.0  for  the  fall  of  1990,  1991,  and  1992, 
respectively.  “Perseus  4.0  will  contain  between  forty  and  one  hundred  megabytes  of  textual  information  (approximately 
forty  to  one  hundred  volumes  of  text,  or  between  five  and  ten  percent  of  the  surviving  literary  material)  and  several 
thousand  images”  (Crane  and  Mylonas,  1988,  p.  26).  Perseus  5.0  was  scheduled  to  be  released  in  December,  1993  (Crane, 
1990). 

The  User's  Manual  for  the  beta  version  of  Perseus  2.0  announced  that  this  version  of  Perseus  “represents  the 
completion  of  a plan,  first  outlined  in  1985,  to  construct  a large  and  heterogeneous  database  of  materials  — textual  and 
visual  — illustrating  the  Archaic  and  Classical  Greek  world.”  The  future  directions  for  the  Perseus  Project  include 
producing  a version  of  Perseus  for  Windows  and  continuing  to  improve  Perseus  site  on  the  World  Wide  Web 
(http://www.perseus.tufts.edu).  The  Perseus  Project  followed  Crane  to  Tufts  University  in  1993. 
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Figure  2.  The  Perseus  Navigator  in  Perseus  2.0 


Costs 

In  1993,  the  entire  Perseus  package  cost  $350  ($200  for  the  videodisc  and  videodisc  index,  $125  for  the  CD-ROM, 
and  $25  for  the  user’s  guide)  according  to  the  Perseus  Order  Form  available  from  Yale  University  Press  (YUP).  The 
predicted  price  for  the  Comprehensive  version  of  Perseus  2.0  with  four  CD-ROM  set  and  possibly  a videodisc  was  about 
$500  (e-mail  communication,  December  17,  1995).  As  recently  as  1993,  Crane  was  still  hoping  to  move  from  sales  to 
institutions  to  sales  to  individuals  where  the  price  of  Perseus  would  be  kept  below  $50  (Crane,  e-mail  communication, 
November  20,  1993).  However,  there  are  currently  no  plans  to  make  that  vision  a reality. 

Acceptance 

The  initial  positive  reaction  to  Perseus  in  the  late  1980s  in  the  general  literature  about  hypertext  and  hypermedia  was 
no  doubt  aided  by  descriptions  of  special  features  in  Perseus.  Crane  and  Mylonas  (1988)  claimed  that  in  the  first  year  of 
work  that  those  involved  in  the  project  had  “developed  exercises  that  help  the  learner  focus  on  important  aspects”  (p.  27) 
of  the  content,  that  they  “used  animation  to  illustrate  the  progress  of  a battle  and  the  techniques  of  architectural  drawing” 
(p.  27),  and  that  they  have  “employed  maps  as  an  introduction  to  prose  history”  (p.  27).  The  Perseus  Sampler,  which  is 
a pre-Beta  prototype  of  Perseus  1 .0  that  reviewers  like  Hughes  (1988)  actually  used  to  formulate  their  opinions  of 
Perseus,  included  these  three  features. 

These  three  features  were  all  special  projects  developed  in  conjunction  with  the  Perseus  materials  and  their 
descriptions  appear  in  the  footnotes  of  the  1988  Crane  and  Mylonas  article.  Figures  of  sample  screens  also  appear  in  the 
aforementioned  article.  It  is  surprising  that  none  of  these  three  features  appear  in  Perseus  1 .0  since  so  much  space  was 
given  over  to  describing  them  in  the  article.  Those  who  read  the  1988  article,  may  have  had  an  opportunity  to  examine 
the  Perseus  Sampler,  and  did  not  have  a chance  to  examine  Perseus  1 .0  might  erroneously  assume  that  these  three 
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features  were  an  integral  part  of  the  commercial  release  of  Perseus  and,  thus,  have  an  inflated  opinion  of  the  capabilities 
of  Perseus. 

Writing  for  the  Bryn  Mawr  Classical  Review,  Eiteljorg,  Hamilton,  O’Donnell,  Pearcy,  and  Wiltshire  (1992)  noted: 
“The  task  of  Perseus  is  teaching.  Arrangement  of  information  forms  part  of  teaching,  but  it  is  not  the  same  as  teaching. 
In  order  for  teaching  to  happen,  one  human  intelligence  must  encounter  another.  A student  using  Perseus  is  not  likely  to 
come  away  with  a sense  that  someone  has  intended  that  he  learn  something.’’  Pearcy  in  particular  objected  to  th  \ non- 
biased  approach  to  this  material.  He  said  “Theirs  is  a neutral,  liberal  program  allowing  unguided  links  between 
miscellaneous  data  ...  and  the  decision  to  avoid  prescriptive  links  between  data  may  be  a weakness  in  Perseus.”  Although 
problems  with  the  operation  of  Perseus  were  noted  and  the  instructional  design  decisions  were  posed,  the  reviewers 
complemented  the  product  profusely.  Even  so,  Crane  and  unnamed  others  (1992c)  responded  at  length  to  the  negative 
points  in  the  review. 

Examples  of  the  Perseus  Hypermedia  User  Interface 

Crane  and  Mylonas  described  the  user  interface  through  examples.  They  said  if  a user  sees  the  name  “Pindar,”  then  he 
or  she  should  be  able  to  call  up  a description  of  that  name.  Perseus  should  allow  students  to  “jump”  from  text  using 
unknown  geographic  names  to  maps  illustrating  the  location  and  nature  of  those  names.  Perseus  should  allow  students  to 
access  literary  references  by  having  a window  open  which  displays  the  work  in  question.  Maps  should  appear  in 
chronological  sequence  to  show  events  of  a period  one  after  another.  Sound  should  be  used  to  convey  the  flavor  of  Greek 
lyric  poetry  or  music  (Crane  and  Mylonas,  1988). 

The  first  part  of  the  hypermedia  design  interface  described  above  which  allows  non-linear  access  of  information  does 
appear  in  the  current  version  of  Perseus  through  a pull-down  menu  called  “Links.”  The  atlas  is  not  fully  implemented  in 
Perseus  1.0  to  allow  for  the  navigation  described  above.  Except  for  the  voice-over  commentary  on  the  videodisc,  sound  is 
not  a part  of  Perseus  1.0.  The  beta  version  of  Perseus  2.0  has  a completely  redesigned  atlas,  but  the  other  features 
mentioned  above  have  not  been  implemented. 


Project  Evaluation 

The  original  Perseus  Project  proposal  included  plans  for  evaluation  and  change.  The  Perseus  Project  Evaluation  team 
believed  that  the  characteristics  of  Naturalistic  Inquiry  (NI)  make  it  particularly  appropriate  to  evaluate  the  complex, 
multifaceted  Perseus  environment  (Neuman,  1991).  The  interview  questionnaires  were  open-ended.  A synopsis  of  a 
student  questionnaire  follows: 

1 . How  would  you  evaluate  the  overall  effectiveness,  attractiveness,  and  usefulness  of  Perseus? 

2.  How  easy  was  it  for  you  to  navigate  in  Perseus? 

3.  How  easy  is  it  for  you  to  learn  with  Perseus? 

4.  Do  your  instructors  use  Perseus  effectively?  Explain. 

5.  What  can  you  suggest  to  improve  Perseus  as  an  instructional  and/or  research  tool? 

6.  What  else  is  important  for  us  to  know  about  what  you  think  and  want  you  do  because  of  Perseus? 

The  four  areas  included  in  evaluation  reports  were  teaching,  learning,  system,  and  content.  Perhaps  Neuman’s  most 
telling  reaction  to  the  evaluation  was  the  following: 

Observations  and  interviews  suggest  that  one  of  the  most  significant  contributions  to  the  Perseus  Project  is  the 
development  of  pedagogical  strategies  that  capitalize  on  the  hypermedia  environment.  Usually,  these  seem  to  arise 
intuitively  from  the  staffs  teaching  experience  and  expertise  rather  than  from  the  theories  and  procedures  of 
systematic  instructional  design.  Several  instructors,  for  example,  came  upon  the  idea  of  using  tools  outside  Perseus 
to  create  dynamic  outlines  that  serve  both  as  organizers  for  lectures  and  as  vehicles  for  moving  in  and  out  of  the 
Perseus  databases  in  both  planned  and  spontaneous  ways.  (p.  245) 

The  bottom  line  here  was  that  Neuman  had  identified  the  need  for  improving  the  instructional  design  of  Perseus.  At 
the  very  least,  copies  of  the  materials  developed  by  these  instructors  who  used  Perseus  ot  beta  test  sites  should  be 
included  with  the  purchase  of  Perseus  1.0  until  Perseus  2.0  is  released.  The  instructional  design  of  subsequent  releases  of 
Perseus  should  be  improved  to  respond  to  the  problems  encountered  by  the  instructors  cited  above.  Subsequent  versions 
of  Perseus  should  rely  more  on  sound  instructional  design  that  makes  the  product  more  user  friendly  than  on  creative 
instructors  who  are  willing  to  develop  auxiliary  materials  to  use  the  hypennedia  aspects  of  Perseus  effectively. 
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Evaluation  Results 

The  results  of  a three-year  evaluation  of  Perseus  were  contained  in  lengthy  article  by  Marchionini  and  Crane  (1994) 
in  the  A CM  Transactions  on  Information  Systems.  The  evaluation  touched  on  a number  of  instructional  design  issues. 
However,  the  evaluation  was  summative,  not  formative,  in  nature.  The  changes  suggested  for  improvement  were  more 
likely  to  target  the  learner  than  the  product. 

The  evaluation  effort  described  here  involves  a complex  system  (Perseus)  based  »*pon  a new  technology  (hypermedia) 
applied  to  abstract  goals  (finding  relevant  information,  learning  how  to  think  critically).  The  overall  evaluation  effort 
aims  to  inform  general  hypermedia  application  design,  develop  human-computer  interaction  and  information-seeking 
behavior  theories,  and  add  to  our  understanding  of  learning  and  teaching.  (Marchionini  and  Crane,  1994,  p.  6) 

Implicit  Links 

In  a section  called  “Design  Decisions,”  the  Marchionini  and  Crane  article  explained  that  the  philosophy  that  guided 
the  creation  of  Perseus  was  “most  strongly  represented  in  the  decision  to  use  primarily  use  implicit  rather  than  explicit 
hypermedia  links”  (p.  12).  This  was  done  in  order  to  “make  it  possible  for  the  user  to  browse  the  database  flexibly  and 
broadly”  (p.  6).  Hughes  (1988)  explained  that  an  implicit  link  was  basically  a keyword  approach  to  information  retrieval 
where  clicking  on  a keyword  called  up  information  in  the  database  that  was  associated  with  that  word.  In  Perseus,  the 
traditional  keyword  notion  has  been  extended  to  include  such  non-text  objects  as  maps,  sculpture,  coins,  urns,  drawings, 
and  archaeological  sites.  While  I applaud  this  particular  design  feature  of  Perseus,  it  should  be  improved  by  providing 
local  and/or  global  maps  to  provide  the  user  with  a sense  of  location. 

Marchionini  and  Crane  gave  another  reason  for  using  implicit  links:  to  create  a situation  where  “the  indexes  and 
tools  are  specific  to  the  underlying  databases  but  not  integrated  with  them.  Thus,  files  containing  text  or  graphics  were 
distinct  from  the  implicit  links  and  can  be  managed  and  transported  separately”  (p.  13).  This  decision  possibly  made  it 
easier  to  write  the  programming  code  necessary  to  access  the  database,  rather  than  on  using  sound  instructional  design. 
One  instructor  estimated  that  it  took  him  20  to  25  hours  “just  to  develop  that  one  exercise  because  I had  to  eyeball  all  the 
pictures  to  make  sure  I was  choosing  the  right  one”  (Marchionini  and  Crane,  p.  27). 

This  aforementioned  decision  to  use  links  which  isolate  the  databases  for  images  and  storing  images  on  the  CD- 
ROM  and  videodisc  means  that  the  images  take  several  seconds  to  appear.  Akseyn,  McCracken,  and  Yoder  asserted  in 
their  1988  article  entitled  “KMS:  A Distributed  Hypermedia  System  for  Managing  Knowledge  in  Organizations”  that  to 
be  effective,  hypermedia  systems  must  execute  linking  with  very  short  response  times  of  less  than  one  second. 

Mechanical  Advantage 

Marchionini  and  Crane  pointed  out  a salient  but  sometimes  overlooked  or  unmentioned  feature  of  such  a product  as 
Perseus.  They  called  it  the  “mechanical  advantage”  of  using  Perseus  to  save  the  time  normally  spent  time  traveling  to  and 
from  a library  or  museum,  locating  books  or  objects,  turning  pages,  and  visually  scanning  texts  and  lists  for  possibly 
relevant  passages.  Of  course,  the  other  side  of  the  coin  is  that  students  may  not  have  easy  access  to  an  institutional  copy 
of  Perseus  and,  once  they  get  access,  the  higher  cognitive  demands  of  interacting  with  a hypermedia  product  may  prevent 
some  students  from  accessing  the  necessary  information  and  making  the  required  connections  between  various  databases 
in  Perseus.  Hence,  there  was  and  is  a need  to  improve  the  instructional  design  for  Perseus  in  order  to  allow  users  make 
more  effective  use  of  this  “mechanical  advantage”  that  Perseus  provided.  The  creators  of  Perseus  might  well  have  had  this 
mechanical  advantage  in  mind  because  of  students’  complaints  that  studying  the  classics  meant  spending  more  time 
finding  information  than  using  it. 

Classical  Content 

For  one  not  versed  in  the  arcane  content  of  Perseus  nor  in  the  type  of  assignments  made  in  the  classics,  the  studies 
presented  in  the  Marchionini  and  Crane  evaluation  article  to  test  Perseus  provide  an  insight  into  the  study  of  the  classics. 
There  may  be  to  be  a direct  connection  between  the  information  stored  in  Perseus  and  way  in  which  the  access  was 
designed  and  the  very  nature  of  studying  the  classics.  As  such,  since  this  is  such  a specialized  body  of  knowledge  and 
access  design,  Perseus  may  not  be  the  model  that  works  in  other  academic  areas,  as  its  creators  had  envisioned.  The 
studies  cited  in  the  Marchionini  and  Crane  article  have  been  selected  based  on  what  is  typically  expected  in  studying  the 
classics.  They  show  that  Perseus  generally  allows  students  to  perform  as  well  as  and  sometimes  better  than  those 
students  using  traditional  methods  of  access  to  this  information.  However,  there  is  no  reason  to  suppose  that  a 
hypermedia  design  that  supports  knowledge  acquisition  in  Greek  civilization  will  necessarily  transfer  to  other  academic 
areas  where  less  rigid  or  different  standards  and  methods  of  study  are  used. 
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Student  Performance 

The  second  conclusion  of  the  Marchionini  and  Crane  article  is  worthy  of  special  attention: 

Perseus  was  not  found  to  change  overall  student  performance  on  traditional  representations  of  critical  thinking 
such  as  translations  and  essays.  The  mechanical  advantages  offered  by  early  implementations  of  Perseus  were  not 
alone  sufficient  to  produce  superior  translations  or  essays.  Perseus  did,  however,  allow  some  students  to  produce 
superior  arguments,  (p.  25) 

The  news  for  Perseus  supporters  was  not  all  bad.  First,  students  using  Perseus  did  not  do  worse  than  those  using 
traditional  materials.  In  fact,  they  did  just  as  well  if  not  better.  Given  that  library  access  is  moving  in  the  direction  of 
CD-ROMs  and  other  such  interactive  storage  mediums,  students  in  the  future  will  come  to  Perseus  better  prepared  to  take 
advantage  of  its  hypermedia  storage  and  retrieval  capacities.  As  for  instructors,  the  aging  professoriate  will  eventually  be 
replaced  by  computer  literate  faculty  who  will  better  equipped  to  take  advantage  of  Perseus  in  the  classroom.  Second, 
student  and  faculty  performances  will  improve  if  the  instructional  design  of  Perseus,  especially  with  the  visual  interface 
presented  to  users,  is  improved. 


The  Status  of  Perseus 

Perseus  2.0  has  not  yet  been  released  although  release  dates  as  early  as  December,  1993,  had  been  promised  in  the 
past.  From  the  introductory  comments  in  the  beta  version  of  the  Perseus  2.0  User  Manual,  the  managers  stated  that 
Perseus  has  achieved  the  goals  set  out  in  the  original  conception  of  the  project.  In  this  author’s  view,  the  interface  design 
of  the  beta  version  of  Perseus  2.0  shows  no  change  in  the  design  philosophy  that  guided  the  creation  of  Perseus. 

Although  the  support  strategies,  such  as  a newsletter,  a web  site,  and  internet  access  to  experts,  continue  to  evolve,  none 
of  the  basic  navigation  and  interface  design  deficiencies  of  the  Perseus  1.0  were  addressed  by  Perseus  2.0. 

Design  Recommendations 

The  list  of  suggestions  below  is  not  meant  to  be  a definitive,  but  it  is  indicative  of  what  design  considerations 
should  be  addressed  in  order  to  improve  the  Perseus  interface, 

I * Redesign  the  Gateway  (see  Figure  1)*  Call  the  2-column  Gateway  either  a temple  (which  is  what  the  5- 
column  icon  with  pediment  looks  like  on  the  “navigator”  bar)  or  a library  (since  the  gateway  is  really  a menu 
repository).  If  the  gateway  is  a metaphor  for  entering  these  various  areas  of  Greek  culture,  a temple  or  library 
would  be  a better  metaphor  since  the  gateway  metaphor  does  not  imply  storage.  When  this  author  encountered 
“dead-ends”  in  Perseus  and  used  the  Gateway  button  to  extricate  himself,  his  frustration  in  the  sense  of  the 
gateway  metaphor  was  heightened  since  he  was  so  lost  he  was  forced  to  re-enter  Perseus  by  the  main  gate.  To 
this  author,  returning  to  the  temple  or  library  where  materials  are  stored  is  far  less  threatening. 

2 . Provide  local  and/or  global  maps  (browsers)  so  that  the  user  has  a sense  of  location  in 
order  to  navigate  more  effectively.  Perseus  should  be  ready  to  suggest  to  the  disoriented  user  possible 
paths  to  follow  from  the  present  screen.  A graphic  environmental  map  should  be  provided  which  is  based  on 
some  concept  or  visualization  familiar  to  the  user  (Tolhurst,  1992). 

3 • Use  a consistent  navigation  interface  for  the  various  kinds  of  storage.  The  navigation  buttons 

are  placed  inconsistently  at  varying  screen  locations  for  various  kinds  of  data.  The  interface  should  be  consistent. 
The  location  of  important  text  and  buttons,  selection  methods,  text  fonts  and  styles,  and  window  layout  should 
be  consistent  in  all  parts  of  the  interface  (Cates,  1992;  Marchionini,  1991). 

4 . Have  dedicated  areas  for  frequently  used  functions  such  as  navigation  (see  Figure  2). 

Perseus  should  be  redesigned  to  take  advantage  of  the  larger  screens  on  today’s  computers.  There  is  room  to 
design  a consistent  user  interface  that  docs  not  pop  up  all  over  the  screen  with  each  new  type  of  database  and 
does  not  get  covered  when  subsequent  windows  open. 

5 . Use  color  throughout  the  interface.  Not  only  is  color  pleasing  to  the  eye,  it  helps  learners  differentiate 
between  various  fields  and  objects  at  a glance.  Certain  functions  and  classifications,  such  as  navigation  and 
collections,  could  be  color  coded  in  a consistent  manner. 

6 . Redesign  the  links  so  that  one  can  browse  through  the  images  using  the  arrow  keys.  Due  to 

the  linkage  design  used  to  make  the  databases  transportable,  one  cannot  browse  through  the  databases  of  stored 
images.  One  must  painfully  call  each  one  up  by  name  from  a menu.  Navigation  should  be  simple,  intuitive,  and 
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consistent  throughout  the  system  (Kearsley,  1988).  Introduce  backwards  linking  so  that  images  can  be  easily 
connected  to  their  descriptions  that  may  have  been  closed  or  covered. 

7.  Avoid  screen  clutter  with  list  after  overlapped  list  appearing  on  the  screen.  Program  Perseus 
for  the  larger  monitor  screens  now  available  so  that  the  images  appear,  as  much  as  is  possible,  in  non- 
overlapping positions.  Since  image  credits  arc  rarely  used  and  a nuisance  to  deal  with,  offer  the  user  an 
opportunity  to  set  the  system  to  “no  credits.”  Since  visual  layout  is  very  important,  screens  should  be  laid  out 
so  that  they  can  be  grasped  perceptually  (Kearsley,  1988). 

8 . Cut  down  on  the  use  of  so  many  lists  and  menus.  If  possible,  create  visual  interfaces  to  allow  the 
user  to  use  the  navigator  more  intuitively.  Expert  users  should  be  given  the  ability  to  move  through  layers  of 
menus  quickly  (Marchionini,  1991).  Add  keyboard  shortcuts  for  frequently  used  menu  items  to  save  time. 

9.  Give  the  choices  in  menu  lists  more  interesting  or  descriptive  names.  A list  such  as  vase  1, 
vase  2,  vase  3 or  poem. ,1,-poem  2,  poem  3,  etc.  gives  the  user  no  clue  as  to  what  they  are  choosing.  Users 
might  have  complete  access  to  “jump”  anywhere  they  like  in  Perseus,  but  these  lists  provide  no  guidance  as  to 
where  to  jump.  Thus,  time  is  wasted,  and  frustration  may  be  the  result. 

10.  Design  the  screen  more  effectively  to  indicate  the  hierarchical  nature  of  the  menus  that 
appear.  Reprogram  so  that  images  of  objects,  sites,  and  menus  pop  onto  the  screen  in  a more  consistent 
manner.  Given  the  larger  screen,  a column  or  row  of  icons  might  be  used  (like  the  toolbar  in  the  latest  versions 
of  Microsoft  Word)  in  order  to  replace  the  lengthy  “Links”  and  “Perseus”  pull-down  menus. 


What  follows  is  a suggested  interface  design  for  the  main  screen  that  would  take  the  place  of  the  Perseus  Gateway 
(Figure  1).  This  design  takes  into  account  many  of  the  recommendations  made  above.  The  most  significant  improvement 
is  that  the  navigation  controls  now  present  in  the  Perseus  Gateway  do  not  disappear  when  a user  begins  to  use  the 
program.  There  is  no  longer  a need  to  return  to  the  Gateway  controls  when  a “deadend”  is  encountered  since  these  controls 
are  omnipresent. 


Suggested  Interface  Design 
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Figure  3.  Suggested  Interface  Design  to  Replace  the  Perseus  Gateway 
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The  HyperCard  menu  selections  have  been  deleted  along  the  top  of  the  screen,  but  the  three  pull-down  menus  (Sites, 
Links,  and  Options)  dedicated  to  Perseus  activities  have  bee.i  retained.  The  nebulous  “Perseus”  puli-down  menu  name  has 
been  replaced  by  “Options.”  Since  the  ‘Temple”  icon  on  the  Perseus  Navigator  (Figure  2)  is  no  longer  needed  to  return  to 
the  Gateway,  that  icon  has  now  been  used  to  “Go  Home”  in  place  of  the  modem  bungalow  that  appears  so  out  of  place 
architecturally  speaking  in  Perseus  1 .0  and  the  beta  version  of  Perseus  2.0.  The  infrequently  used  “Settings”  and  “Info” 
icons  have  been  given  less  prominence  in  this  design. 

The  icons  for  the  major  categories  in  the  database  that  previous  appeared  in  the  center  of  the  Gateway  now  appear 
down  the  left-hand  side  of  the  screen  in  a dedicated  area  that  is  always  available.  For  clarity,  the  icon  for  “Essays  & 
Catalogs”  has  been  changed  to  a different  icon  than  the  one  for  “Historical  Review.”  The  use  of  the  same  icon  for  both 
classifications  in  Perseus  1.0  and  the  beta  version  of  Perseus  2.0  could  be  confusing  and  may  lead  even  an  experienced 
user  to  click  on  the  wrong  one.  The  Atlas  icon  now  appears  next  to  the  pull-down  menu  used  to  support  its  usage. 

All  the  navigation  tools  and  devices  that  allow  the  user  to  move  through  the  database  are  located  together  across  the 
bottom  of  the  screen  and  available  at  all  times.  Names  have  been  added  to  the  buttons  that  previously  appeared  in  the 
Perseus  Navigator  in  an  unnamed  form.  The  Perseus  “Footprint,”  the  double-ended  arrow  icon  with  the  maze  in  the 
middle,  has  been  lengthed  and  lebeled  so  that  users  will  be  reminded  of  its  three-fold  functionality.  The  “plus  sign”  in  the 
center  reinforces  the  notion  that  one  can  “add”  stops  on  a pathway  by  clicking  on  it. 

The  “Recent”  button  with  its  “eye”  icon  permits  the  user  to  view  the  last  forty-two  screens  used  in  HyperCard.  For 
those  unfamiliar  with  this  HyperCard  feature,  a user  need  only  click  on  the  thumb-nail  version  of  a previously  visited 
screen  for  the  application  to  jump  to  that  screen.  The  reason  for  this  addition  to  the  navigation  is  that  the  link  structure 
in  Perseus  does  not  premit  the  “Return”  button  with  its  “curved  arrow”  icon  to  work  properly.  The  Perseus  1.0  and  beta 
version  of  Perseus  2.0  User’s  Manuals  recognize  that  fact  and  suggest  that  users  use  the  pull-down  manu  in  HyperCard  to 
access  the  “Recent”  command  when  they  find  they  are  at  a deadend  because  the  “Return”  button  does  not  work.  The 
repeated  use  of  the  Return  button  often  leads  to  infinite  loops  where  the  same  screens  appear  again  and  again.  The  Recent 
button  allows  the  user  to  break  the  cycle  in  order  to  continue  working  in  the  database  at  a point  before  the  impasse 
occurred. 


Conclusion 

A larger  consideration  than  the  hypermedia  design  suggestions  listed  above  is  the  fact  that  the  beta  version  of  Perseus 
2.0  has  not  made  any  significant  instructional  design  changes  from  Perseus  1.0.  Rather  than  expecting  learners  to  adapt 
to  the  eccentricities  of  Perseus  design,  the  design  of  Perseus  should  have  adapted  to  the  needs  of  learners.  More  people 
need  to  be  allowed  or  encouraged  to  become  involved.  Instructional  design  experts  need  to  be  consulted  to  contribute  their 
expertise.  Given  the  advantage  of  heavy  funding,  Perseus  will  not  be  challenged  in  the  near  future.  However,  the  chosen 
direction  of  the  Perseus  was  to  get  bigger,  not  better.  Except  for  the  atlas,  instead  of  improving  the  interface  design,  they 
kept  what  they  had  and  added  more  features.  The  goal  of  attracting  students  to  buy  their  own  copies  of  Perseus  will  not  be 
achieved  by  retaining  an  interface  design  that  is  cumbersome  to  use. 

As  for  the  Perseus  Project  being  used  as  model  for  other  academic  areas,  its  application  in  that  direction  is  limited  by 
two  problems:  (1)  the  fact  that  only  one  version  of  Perseus  has  appeared  in  seven  years,  and  (2)  the  price  of  Perseus. 
Relative  to  problem  (1),  the  CDI  (Compact  Disc  Interactive)  product  currently  being  offered  by  Phillips  (that  included  a 
Compton's  Pictured  Encyclopedia  on  CD-ROM  for  $500  total)  should  make  it  clear  that  commercial  vendors  are  not 
going  to  sit  back  and  wait  for  the  members  of  academe  to  decide  how  to  design  informational  products.  Manufacturers 
will  simplify  produce  and  sell  what  they  can  as  quickly  and  economically  as  they  can.  When  the  dust  clears,  the  best 
sellers  (the  market  place)  will  have  determined  what  many  will  consider  to  be  the  best  design  characteristics  for  hypertext 
and  hypermedia  systems. 

Relative  to  problem  (2),  all  the  faculty  members  who  teach  calculus  at  my  institution  received  complimentary  copies 
of  a CD-ROM  with  an  entire  calculus  book  stored  on  it.  The  point  is  that  vendors  are  willing  to  give  away  CD-ROMs 
on  mathematics  to  induce  faculty  members  to  adopt  their  materials.  Compare  that  to  the  current  $150  price  tag  on  the 
Perseus  1.0  CD-ROM  with  user’s  guide  or  $350  for  the  entire  package  which  includes  the  videodisc.  Some  innovative 
marketing  ideas  should  be  used  to  get  the  Perseus  Project  moving  again  after  its  promising  start  over  seven  years  ago. 

Before  Perseus  can  achieve  the  status  of  being  a design  model  for  other  subjects,  Perseus  must  first  incorporate 
design  changes  to  make  the  product  more  user  friendly  by  being  more  visually  interesting  while  maintaining  a consistent 
screen  design.  The  navigational  problems  alone  make  this  design  impractical  for  most  learners  except  for  the  most 
motivated  who  will  learn  in  spite  of  the  instructional  materials  used.  At  this  stage  in  its  development,  Perseus  is  an 
adequate  model  for  esoteric  academic  areas  where  its  “mechanical  advantage”  to  access  encyclopedic  information  databases 
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can  be  exploited.  However,  Perseus  1 .0  and  the  beta  version  of  Perseus  2.0  do  not  exhibit  an;  fecial  design 
characteristics  that  would  make  it  a clear  choice  over  other  products  which  are  currently  being  developed  for  mainstream 
instructional  applications. 
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In  1986,  when  the  concept  of  a graphical  user  interface  (GUI)  was  still  new,  Norman  challenged  designers  of 
computer-based  programs  to  heed  the  needs  of  the  user.  He  contended  that  interface  design  should  stand  separate  from  the 
needs  of  the  system;  in  fact,  the  system  should  be  “transparent”  to  the  user.  As  far  as  the  user  is  concerned,  Norman 
argued,  the  interface  is  the  system. 

Ten  years  later,  GUIs  are  everywhere.  Apple  Macintosh  and  Windows  desktop  GUIs  dominate  the  market. 
Software  developers  vie  to  see  who  can  release  the  “more  graphical”  program.  Users  now  recognize  icons  and  buttons  as 
staples  of  most  interfaces,  and  many,  especially  the  young,  have  grown  increasingly  sophisticated  in  their  taste  for 
attractive  interface  design. 

Utilizing  Metaphors  in  the  Graphical  User  Interface 

Laurel  (1993)  contended  that  “an  interface  is  not  simply  the  means  whereby  a person  and  a computer  represent 
themselves  to  one  another;  rather  it  is  a shared  context  for  action  in  which  both  are  agents”  (p.  4).  As  graphics  have 
become  increasingly  more  sophisticated  and  dominant  in  the  human-computer  interface,  they  offer  users  opportunities  to 
process  and  interact  with  information  in  ways  likely  to  be  richer  than  text  alone.  Brown  (1988)  asserted  that  an  important 
advantage  of  graphical  presentation  is  in  increasing  the  rate  at  which  users  can  process,  understand,  and  respond  to  a 
display.  But  the  presence  of  graphics  alone  does  not  assure  Norman’s  “transparent”  interface.  In  fact,  including  graphics 
may  actually  make  the  interface  less  comprehensible.  Increasingly  complex  graphics,  like  the  sirens  of  mythological 
fame,  lure  us  ever  closer  to  foundering  and  failure.  But,  if  graphics,  in  and  of  themselves,  do  not  enhance 
comprehensibility  and  help  users  understand  how  to  make  the  most  of  a program’s  promise,  what  does?  Perhaps 
metaphorical  interfaces  do. 

A metaphor  is  a figure  of  speech  which  applies  its  symbolic  images  to  a subject,  to  improve  our  understanding 
of  that  subject  by  linking  the  familiar  with  the  unknown.  Paivio  (1979)  argued  that  a metaphor  acts  as  a vehicle  when 
used  in  conjunction  with  another  object,  a topic ; to  impart  a similarity  or  relationship  which  helps  render  the  new 
knowledge  we  are  acquiring  intelligible.  Lakoff  and  Johnson  (1980)  asserted  that  metaphors  are  a pervasive  part  of  our 
speech  and  thought,  used  to  understand  and  experience  one  kind  of  thing  in  terms  of  another.  Erickson  (1990)  agreed, 
suggesting  that  metaphors  are  invisible  webs  of  terms  and  associations  that  underlie  the  way  we  communicate. 

Metaphors  can  convey  rich  images  and  chunks  of  information  that  hold  more  meaning  than  the  literal  definition 
or  singular  implication  of  the  phrase  used  as  a vehicle.  When  people  perceive  the  presence  of  a metaphor,  they  explore  a 
range  of  potentially  applicable  emotions,  sensations,  and  mental  images.  This  range  and  its  components  appear  different 
for  each  individual,  but  are  tied  in  some  way  to  individual  experiences  and  imaginations,  helping  to  build  individual 
cognitive  models  (Lakoff,  1987;  Ortony,  1979;  Petrie,  1979).  In  fact,  Black  (1979)  contended  that  “every  metaphor  is  the 
. tip  of  a submerged  model”  (p.  31).  But  the  model's  existence  does  not  mean  that  the  user  must  accept  the  complete 
package  of  implications  for  the  metaphor  to  work.  Rather,  Black  argued,  the  perceiver  selects,  organizes,  projects,  and 
suppresses  portions  of  the  metaphor  as  needed  to  facilitate  comprehension  and,  at  the  same  time,  codes  both  the  topic  of 
the  metaphor  and  the  vehicle  into  long-term  memory  (see  also  Mountford,  1990;  Paivio,  1979). 

According  to  Rieber  (1994),  graphics  in  the  computer  interface  can  take  a number  of  forms,  particularly  for 
computer-based  instruction.  They  can  be  arbitrary,  like  a graph  which  illustrates  a logical  or  conceptual  relationship. 
They  can  be  representational,  like  an  illustration  of  two  people  having  an  argument.  Or  they  can  be  analogical,  as  when 
an  image  of  two  knights  squaring  off  in  a jousting  match  represents  two  people  in  an  argument.  Analogical  graphics 
might  also  be  described  as  visual  metaphors.  Visual  metaphors  in  interfaces  can  share  the  same  properties  as  the  ones  we 
use  in  speech,  according  to  Dent  and  Rosenberg  (1990).  The  authors  argued  that  as  a verba)  metaphor  acts  as  a vehicle  to 
describe  or  explain  a less  familiar  topic,  a visual  metaphor  depicts  a vehicle  object  using  properties  that  are  recognizably 
related  to  the  topic  object.  As  illustrated  in  the  clip  art  cartoon  of  Figure  1,  the  most  important  properties,  those  which 
exemplify  the  connection  between  vehicle  and  topic,  are  emphasized  in  the  visual  metaphor.  The  vehicle,  an  office 
worker,  is  revealed  by  just  a few  properties:  a desk,  chair,  and  typewriter.  His  dilemma  is  indicated  by  the  sweat  pouring 
off  his  brow  and  the  canoe  heading  over  the  lai!s.  The  state  of  “teetering  on  the  brink”  is  a metaphor  for  the  topic, 
apparently  a deadline  that  is  fast  approaching. 

Graphical  user  interfaces  can  integrate  images  more  fully  than  just  an  occasional  spot  illustration.  They  can 
employ  icons,  small  graphical  representations  to  indicate  interactive  objects, 
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actions,  or  concepts.  GUIs  can  also  utilize  large  graphical  images  to  represent 
something  other  than  the  actual  computer  screen.  Those  large  graphical  images 
can  be  integral  to  the  meaning  and  the  action  of  a computer  program,  and  they 
can  be  metaphorical  in  nature. 


What  a metaphorical  GUI  has  to  offer 

Engagement.  Laurel  (1993)  contended  that  an  interface  metaphor  can  serve  as  a “mimesis,”  a functional 
representation  that  encourages  the  user  to  become  engaged,  to  participate,  to  willingly  suspend  his  or  her  disbelief  as  we 
do  for  movies  and  plays.  Such  a metaphor,  wliich  Keller  (1983)  called  a metaphoric  organizer,  can  grab  the  user’s 
attention  while  acting  as  an  advance  organizer  for  the  actual  content,  in  a sense  planting  clues  for  the  user  to  pick  up, 
developing  a sense  of  expectation.  Keller  argued  that  using  such  an  organizer  can  help  students  relate  a more  abstract 
concept  to  a familiar,  concrete  one,  build  curiosity,  and  add  to  an  effective  learning  experience. 

Orientation.  A metaphorical  interface  may  help  users  learn  how  to  use  a program  or  system.  Unless  we  design  it 
carefully,  however,  an  opposing  disadvantage  might  be  that  the  metaphor  could  break  down  as  the  novice  learner  moves 
toward  expertise,  developing  an  understanding  of  the  system  or  program  that  can  extend  beyond  the  metaphor’s  utility 
(Laurel,  1993). 

Familiarity  and  Transfer . Hammond  and  Allinson  (1987)  asserted  that  a primary  strength  of  metaphors  lies  in 
their  familiarity.  The  authors  developed  and  tested  a “travel  holiday”  metaphor  to  aid  users  in  navigating  a general 
knowledge  base.  Their  results  suggest  that  once  the  user  has  established  the  primary  relationship  between  the  metaphor 
and  the  topic  to  which  it  relates,  secondary  relationships  can  be  more  easily  established  as  the  user  retrieves  familiar 
knowledge  and  actions  from  the  metaphor’s  domain.  They  termed  this  “piggy-backing.”  The  authors  argued  that  metaphor 
use  reduces  the  need  for  inference  and  logical  thought.  Plausible  but  erroneous  mappings  of  the  metaphor  to  the  topic 
can,  however,  occur  if  the  designer  has  not  considered  such  mappings  and  their  causes  in  advance.  A designer  who  has  so 
considered  potential  mismappings  should  be  able  to  suggest  additional  useful  aspects  of  the  metaphor  to  decrease  the 
likelihood  of  their  occurring. 

Bielenberg  (1993)  concurred,  asserting  that  visual  metaphors  in  GUI  instructional  programs  can  be  one  of  the 
most  powerful  ways  to  transfer  meaning.  He  used  as  an  example  a ‘Test  Drive”  program,  designed  with  a metaphor  of  a 
car  dashboard  and  driver  controls  to  convey  technical  information  about  the  capabilities  of  digital  video  interactive  (DVI). 
Bielenberg  contended  that  a user  who  is  familiar  with  automobiles  brings  to  the  instructional  program  a mental  model  of 
the  functionality  of  the  car’s  dashboard  and  controls.  The  designer  offers  a conceptual  model,  a structure  composed  of  the 
content  to  be  taught,  the  task  to  be  performed,  or  the  information  to  be  displayed.  The  purpose  of  the  interface,  according 
to  the  author,  is  to  encourage  the  user’s  mental  model  to  conform  to  the  designer’s  conceptual  model  quickly, 
completely,  and  painlessly.  He  asserted  that  visual  metaphors  can  support  that  process  and  urged  designers  to  analyze  the 
needs  of  the  learner  and  the  structure  of  the  content  when  choosing  a visual  metaphor. 

Coherence.  Providing  coherence  is  another  advantage  of  integrating  a metaphor  into  the  interface,  according  to 
Laurel  (1993).  If  the  metaphor  is  well-chosen,  all  of  the  elements  of  a program  or  system  should  cohere  naturally,  for 
example,  as  files  in  the  Macintosh  desktop  fit  into  folders.  Cates  (1994)  refers  to  this  as  the  presence  of  complementary 
auxiliary  metaphors,  in  this  case  to  accompany  the  underlying  metaphor  of  a desktop.  Of  course,  limitations  of  this 
metaphor’s  implementation  become  evident  when  users  begin  to  look  for  other  “items”  they  expect  to  see  on  the  Mac 
desktop,  like  staplers  and  paper  clips,  and  fail  to  find  that  functionality,  or  when  they  find  a trash  can  on  top  of  their 
desk.  This  latter  example  is  an  instance  of  a confounding  auxiliary  metaphor  (Cates,  1994). 

Implementing  a Metaphor  to  Deliver  Instruction  on  Stress  Management 

The  pervasiveness  of  stress  in  our  lives  makes  stress  management  a popular  topic.  Selye  (1981),  considered  the 
father  of  modern  stress  theory,  contended  that  stress  is  inevitable.  Without  it  we  would  suffer  from  a lack  of  motivation 
and  drive.  Many  workshops,  books,  and  tapes  exist  on  stress  management,  and  many  techniques  to  reduce  or  cope  with 
stress  have  met  with  varying  degrees  of  success. 

Any  adult  who  remembers  his  or  her  college  years  as  idyllic  and  stress-free  is  probably  exhibiting  a case  of 
selective  memory.  According  to  Benjamin  (1987),  college  students,  particularly  freshmen,  face  many  new 
responsibilities.  Among  them  are  organizing  their  time,  handling  new  social  and  administrative  interactions,  adapting  to 
large  numbers  of  students  who  are  unknown  to  them,  and  often  facing  much  more  rigorous  academic  performance 
standards.  Stress  naturally  accompanies  these  new  responsibilities.  A recent  World  Wide  Web  report  by  the  Counseling 
Center  of  the  State  University  of  New  York  at  Buffalo  (1995)  listed  stress  periods  for  students  by  month.  Every  month  of 
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the  student's  academic  year  held  a bulleted  list  of  at  least  four  stressors  or  stress  reactions  based  on  academic  and  other 

aspects  of  normal  college  life. 

Computer-based  instruction  on  stress  management 

A computer-based  instructional  (CBI)  program  about  stress  management  could  be  quite  large,  since  stress  ideally 
should  be  managed  on  a number  of  levels  - physiological,  cognitive,  emotional,  and  behavioral  (Roskies,  1987).  In  order 
to  narrow  the  field  to  a manageable  size  for  testing  the  implementation  of  a visual  metaphor,  the  researchers  have  chosen 
to  develop  a CBI  lesson  for  college  students  on  1)  the  fundamentals  of  stress,  and  2)  the  role  of  time  management  as  a 
coping  strategy.  The  fundamentals  of  stress  include  building  an  awareness  of  specific  sources  of  stress  for  college 
students,  and  identifying  ways  to  cope  with  stress  by  managing  it  effectively,  utilizing  available  support  mechanisms. 

The  portion  of  instruction  on  time  management  will  attempt  to  validate  a model  developed  by  Brown  (1992).  Brown 
suggested  that  students  first  map  the  time  allotted  for  various  activities  against  the  available  time  in  each  week.  Next, 
they  evoke  memories  and  a visual  representation  of  the  past  semester’s  peak  stress  periods.  Last,  they  practice  ways  to 
handle  moderately  increased  stress  now  in  order  to  reduce  it  significantly  later. 

Description  of  content 

The  instructional  program,  called  “Building  Coping  Skills  on  a Firm  Foundation,”  begins  by  offering  nine 
categories  of  college  student  stress  (Johnson,  1978):  instruction,  competition,  managing  time,  adjusting  to  college, 
administrative  problems,  social  adjustment,  finances,  housing,  and  transportation.  Students  are  informed  about  the 
particular  relevance  of  stress  and  stress  management  to  their  lives.  Students  then  examine  how  to  react  to  the  addition  of  a 
new  stress  factor  and  explore  the  relationship  of  balance  in  the  realms  of  physical  health,  psychological  health,  academic 
obligations,  and  social  life  (including  the  support  of  friends,  faculty,  and  professionals)  to  coping  with  stress.  They  learn 
about  the  forms  that  stress  can  take,  and  add  personal  descriptors  of  physical  and  emotional  signs  of  stress. 

Beginning  a decision-making  exercise  involving  a proposed  new  stress  factor,  students  chart  a weekly  schedule, 
dividing  hours  of  the  week  into  categories  for  sleep,  eating,  classes,  and  other  components.  They  note  the  number  of 
hours  left  for  study  and  compare  them  to  an  ideal.  Given  the  added  stress  factor,  students  practice  coping  by  adjusting 
their  schedules,  assessing  the  effects  of  their  adjustments  on  the  various  components.  Feedback  offers  pros  and  cons  about 
those  adjustments  (for  example,  the  effects  of  inadequate  sleep).  The  program  analyzes  the  results  in  terms  of  a healthy 
balance  of  physical,  academic,  social,  and  psychological  needs  and  provides  a running  stress  assessment  report.  In  a final 
metacognitive  exercise,  students  select  the  ten  ideas  most  useful  to  them  from  a given  list  of  ideas  about  how  to  improve 
time  management  skills  for  study  and  living.  They  select  and  drag  them  to  a report  form,  adapt  them,  add  new  ideas,  and 

print  out  the  resultant  form. 

The  role  of  a metaphorical  GUI  for  this  program 

Engagement.  Many  of  us  view  the  efficacy  of  our  actions  on  a daily,  even  moment-by-moment  basis,  rarely 
taking  the  time  to  step  back  and  take  a long  view.  Most  of  us  feel  that  at  any  given  time  we  are  doing  the  best  we  can. 
Though  stress  is  neither  the  province  of  the  young  nor  the  old,  college  students  can  often  find  themselves  in  difficult  and 
stressful  situations,  compounded  by  their  youth  and  inexperience.  Brown  (1992)  discussed  the  motivational  conflicts 
students  can  face  when  they  confront  at  least  two  simultaneous  choices.  One  is  usually  academic  in  nature,  with  delayed 
reward  (grades)  and  remote  consequences  (positive  or  negative  reinforcement  in  the  classroom,  future  career  choices).  The 
others  are  often  social  activities,  with  immediate  reward  (exercise  leading  tq  a sense  of  physical  well-being,  the  pleasures 
of  food  and  drink,  positive  reinforcement  from  friends).  Given  a natural  tendency  to  be  short-sighted,  some  students  might 
predict  that  an  instructional  lesson  on  stress  management  would  be  dry,  “preachy,  and  an  inaccurate  reflection  of  their 
lives.  Introducing  the  lesson  with  a metaphorical  GUI  before  they  meet  the  lesson’s  content  could  serve  the  purpose  of 
engaging  learners,  as  Laurel  (1993)  argued,  to  participate,  to  buy  in,  and  to  suspend  their  disbelief.  The  metaphor  should 
be  concrete,  comprehensible,  and  familiar  enough  to  all  college  students  that  it  would  not  cause  cognitive  dissonance  nor 
lose  the  audience.  Learners  should  suspend  their  disbelief  on  participating  in  the  “mimesis”  willingly,  not  reluctantly. 

Orientation.  As  with  most  instructional  computer-based  programs,  there  are  uniform  conventions  of  use,  like  a 
stop  or  quit  function,  the  ability  to  navigate,  and  expected  responses  to  data  input.  But  there  are  also  unique  elements  in 
the  functionality  of  any  program.  A metaphorical  GUI  could  help  learners  become  oriented  to  both  predictable  and  unique 
functions  by  placing  all  functions  in  a consistent  context.  For  example,  elements  of  the  metaphorical  interface  could 
indicate  properties  of  use,  such  as  a “report”  in  which  data  can  be  entered,  or  a set  of  binoculars  which  could  magnify  a 
view  or  provide  deeper  insight.  Expected  functions  could  be  portrayed  in  ways  consistent  with  the  metaphor;  for  example, 
a building  construction  site  might  display  its  stop  function  in  the  form  of  an  EXIT  sign  on  the  fence  surrounding  the 


Familiarity  and  Transfer.  We  have  already  noted  that  our  lives  seem  intimately  familiar  to  us.  Yet  life  can  hand 
us  lemons  and,  at  any  given  time,  we  may  have  greater  or  lesser  abilities  to  make  lemonade.  Similarly,  the  subject  of 
stress  and  our  ability  to  cope  may  seem  familiar  territory,  yet  it  may  not  be  as  familiar  as  we  think.  Lakoff  and  Johnson 
(1980),  paraphrasing  Aristotle’s  Rhetoric,  suggested  that  metaphors  are  one  of  our  most  important  tools  for  trying  to 
comprehend  partially  what  cannot  be  comprehended  totally  — our  feelings,  aesthetic  experiences,  moral  practices,  and 
spiritual  awareness.  A metaphorical  GUI  portraying  a familiar  setting,  situation,  or  event  can  offer  a more  concrete 
framework  to  help  learners  acquire  new  ways  to  picture  what  may  not  be  comprehended  totally.  In  effect,  the  metaphor 
can  provide  learners  with  a conceptual  fix  on  what  may  always  be  a moving  target,  as  feelings  change  constantly  and 
stress  levels  ebb  and  flow.  The  designer  can  encourage  learners  to  bring  their  mental  model  of  the  metaphorical  GUI  to 
the  instructional  event,  as  Bielenberg  (1993)  suggested.  The  learner's  mental  model,  or  schema,  should  support  his  or  her 
new  understanding  of  the  topic  if  the  designer’s  conceptual  model  of  the  content  is  mapped  well  to  the  needs  of  the 
program’s  users  and  to  the  metaphorical  GUI. 

A mental  model  may  be  particularly  important  to  the  content  in  this  case  because  the  chosen  lesson  focuses  on 
relational  concepts:  a balance  of  finite  resources  (time,  health,  academic  requirements),  personal  responses  to  stressful 
situations,  and  decisions  affecting  short-  and  long-term  goals.  For  college  students,  stress  can  be  found  in  as  many  as 
nine  categories.  Each  category  can  be  perceived  as  a layer  that  each  student  must  handle,  and  the  weight  of  those  layers 
can  pile  up  intensively  and  with  disastrous  consequences,  should  the  student  not  have  a strong  enough  “foundation”  to 
bear  the  weight.  In  addition,  each  individual  has  a propensity  to  respond  in  a number  of  different  ways.  Some  may  tend  to 
ignore  the  problem  until  they  sink  under  the  weight.  Others  may  put  a temporary  “patch”  on  the  situation,  risking  future 
complications,  while  others  will  stop  and  focus  long  enough  to  assess  the  problem  and  address  it  as  definitively  as 
possible.  In  this  case  a visual  metaphor  could  build  a fitting  mental  model  to  help  learners  transfer  their  new  knowledge 
to  future  stress-related  events  beyond  the  scope  of  this  lesson,  hel-’ig  them  imagine  ways  to  adjust  those  relationships  in 
order  to  successfully  cope  with  new  problems. 

Coherence.  Finally,  a metaphorical  GUI  can  provide  this  stress  management  program  with  visual  elements  that 
tie  the  metaphor  to  the  content  throughout  the  lesson,  providing  a sense  of  coherence  and  reinforcement  for  the 
functionality  of  the  learner’s  mental  model  in  representing  the  content.  All  elements  of  the  metaphor  should  logically  and 
coherently  support  the  lesson,  including  the  presentation  of  information,  the  exercises,  feedback,  reports,  and 
opportunities  for  practice.  All  auxiliary  metaphors  should,  therefore,  be  complementary. 

Components  of  the  metaphor 

For  this  program,  “Building  Coping  Skills  on  a Firm  Foundation,’*  the  primary  metaphor  is  a building 
construction  site.  The  learner  enters  the  program  as  a new  manager  of  building  construction.  The  building,  a new  student 
center  for  the  fictional  Piloton  University,  consists  of  a foundation  and  a number  of  framed-out  stories.  As  the  brief  storv 
line  develops,  you,  the  new  manager,  are  made  aware  by  other  employees  that  your  boss,  Ike  N.  Cope,  has  authorized  the 
addition  of  four  more  layers,  or  stories,  to  the  building.  Figure  2 shows  the  building  under  construction.  The  building 
represents  the  life  of  a student,  supported  by  a foundation  of  physical  health,  and  including  layers,  or  “stories”  of  stress 
factors,  including  academic  requirements,  competition,  financial  pressures,  and  other  components  listed  above. 


The  manager,  of  course,  manages  construction  of  the  edifice,  just  as  learners  manage,  with  greater  and  lesser  degrees  of 
success,  the  components  of  their  lives  as  college  students. 

As  the  introductory  plot  develops,  another  coworker  points  out  that  there  is  a problem  in  the  building’s 
construction.  A crack  has  appeared  in  a support  beam,  and  a connected  beam  has  started  to  bend.  The  coworker  implies 
that  there  may  be  a problem  with  the  foundation,  particularly  if  it  must  support  the  weight  of  additional  floors.  As 
manager  you  are  faced  with  an  immediate  choice:  to  ignore  the  construction  problem,  to  “patch”  it,  or  to  stop 
construction  briefly  (for  everyone’s  safety)  and  plan  the  best  way  to  deal  with  it.  Animation  sequences  and  “faxed” 
feedback  from  the  remote  boss  can  demonstrate  the  results  of  these  choices  to  the  primary  metaphor,  the  building  and  its 
foundation.  As  the  program  moves  from  the  metaphorical  GUI’s  story  to  a more  content-intensive  lesson,  learners  will 
again  face  this  dilemma  as  an  unexpected  theoretical  stressor  is  added  to  their  lives,  forcing  them  to  examine  the  results  of 
the  same  coping  choices:  ignore,  patch,  or  manage  it.  Feedback  is  again  given,  reflecting  the  images  and  language  of  the 
primary  metaphor.  Thus,  the  tools  of  the  metaphor  come  to  be  the  tools  of  use  in  the  realistic  setting,  a form  of  near 
transfer. 

To  help  designers  develop  metaphorical  GUIs  that  effectively  support  instruction,  Cates  (1994)  suggested  a 
method  for  identifying  related  properties  of  a primary  metaphor.  Called  a POPIT  approach,  this  method  identifies  a 
prospective  metaphor’s  properties,  operations,  phrases,  images,  and  types.  A POPIT  approach  can  reveal  bad  news  and 
good  news.  The  good  news  may  be  that  a metaphor  offers  a rich  environment  of  secondary  metaphors,  consonant 
components  to  support  acquisition  of  new  knowledge.  The  bad  news  may  be  that  a component  of  the  metaphor  is 
dissonant,  leading  to  rejection  either  of  that  component  or  -if  it  can’t  be  ignored-  the  whole  metaphor.  However,  the  bad 
news  is  actually  good  news  for  the  learner,  who  will  have  avoided  a potentially  irritating  learning  experience. 

A POPIT  of  the  building  construction  site  metaphor  revealed  a number  of  useful  secondary  metaphors.  One,  a 
blueprint,  shown  in  Figure  3,  indicates  the  four  extra  floors  that  should  be  added  to  the  student  center.  Each  floor  contains 
rooms  or  offices  representing  some  of  the  stressors  typically  faced  by  college  students:  academic  requirements,  financial 
needs,  career,  and  so  forth.  The  top  level,  a recreation  center,  represents  the  need  for  friends  and  relaxation  time,  but  can 
also  represent  a source  of  stress  if  time  dedicated  to  recreational  activities  (and  recovery  time  for  some  of  them)  is  not 
managed  reasonably. 


Figure  3:  Blueprint 


While  the  physical  factors  are  represented  by  the  building’s  foundation,  Figure  4 reveals  auxiliary  metaphors  for 
the  three  ether  factors  of  college  student  life  that  require  balance  in  order  to  manage  stress:  psychological  health,  academic 
obligations,  and  social  life.  “Your”  hand  holding  a pocket  watch  and  a list  of  problems  that  must  be  addressed  represents 
academic  obligations.  Some  workers  at  the  construction  site  represent  auxiliary  metaphors  for  social  life.  The  lower  right 
comer  of  Figure  4 shows  four  construction  workers  helping  each  other  by  carrying  two  beams  between  them, 
representing  the  support  of  friends.  Another  worker  sits  on  a beam  bearing  the  American  flag.  He’s  waving  and  happy, 
motivated  to  do  his  job  well  and  with  pride.  This  worker  is  an  auxiliary  metaphor  for  psychological  health.  Though  this 
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lesson  focuses  primarily  on  time  management  as  a coping  strategy  for  the  strain  of  additional  stressors,  the  other 
elements  are  noteworthy  and  necessary.  To  illustrate  the  relationship  of  all  four  factors  to  each  other,  the  learner  is 
presented  with  a conundrum:  what  if  the  foundation’s  strong,  your  project  is  on  time,  the  teams  are  working  well 
together,  but  a few  workers  are  depressed?  What  if  any  other  one  of  these  factors  is  out  of  kilter9  This  same  kind  of 
conundrum  reoccurs  later  in  the  instruction,  this  time  dealing  with  actual  content.  Reinforcement  is  provided  as  the 


Figure  4:  Additional  Auxiliary  Metaphors 


content  is  tied  to  the  appropriate  metaphoric  images. 

Besides  the  blueprint,  the  workers,  and  the  pocket  watch,  another  set  of  auxiliary  metaphors,  business  forms, 
provides  coherence  to  this  program’s  functionality.  As  mentioned  earlier,  after  students  are  given  a new  stress  factor  they 
are  presented  with  a decision-making  exercise  in  which  they  practice  how  to  manage  their  time.  They  take  a 
comprehensive  look  at  the  hours  they  spend  at  various  activities,  dividing  the  hours  of  the  week  into  categories.  They 
practice  coping  by  adjusting  their  schedules,  and  receive  feedback  in  the  form  of  a stress  assessment  report.  Finally  they 
develop  a report  containing  ideas  about  how  to  improve  their  time  management  -and  thus  coping-  skills.  Doing  this 
exercise  requires  three  forms,  shown  in  Figure  5 below:  one  for  students  to  chart  and  adjust  hours  spent,  one  to  provide 
feedback  to  the  learner,  and  one  for  each  learner’s  personalized  coping  ideas.  As  auxiliary  metaphors,  these  forms  can  take 
the  guise  of  a spreadsheet,  a stress  assessment  report  (a  play  on  words  for  “stress”  as  a strain  to  the  foundation  and 
beams),  and  a memo. 

Visual  reinforcers 

In  addition  to  the  use  of  primary  and  secondary  metaphors,  this  program  uses  color  selectively.  Primarily  black 
and  white,  the  metaphorical  GUT  uses  select  areas  of  color  to  focus  the  learner’s  attention.  For  example,  on  the  screen  in 
which  a coworker  indicates  to  the  manager  that  there  is  a construction  problem,  the  only  item  containing  color  is  a set  of 
binoculars  the  coworker  offers  to  the  manager.  In  Figure  4,  only  the  team  workers,  the  pocket  watch,  and  the  worker  with 
the  flag  are  colorful.  The  learner’s  eye  is  directed  by  color  and  by  text  cues  to  attend  to  selected  portions  of  the  screen. 

This  treatment  has  the  added  benefit  of  rendering  the  computer  program  a more  manageable  size  without  giving  up 
important  design  elements. 
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Figure  5:  Spreadsheet,  Report,  Memo 


The  Field  Test 

Upon  completion,  “Building  Coping  Skills  on  a Firm  Foundation”  will  be  tested  with  a sample  population  of 
approximately  150  college  students,  both  freshmen  and  upperclassmen,  to  determine  if  the  metaphorical  graphical  user 
interface  contributes  to  the  students*  understanding  of  the  relationship  of  time  management  to  coping  with  stress.  The 
program  will  be  tested  in  two  formats:  half  of  the  sample  will  utilize  the  program  with  content  tied  into  a metaphorical 
GUI,  while  the  other  half  will  use  the  format  containing  the  stress  management  content  without  a metaphorical  GUI. 

“Building  Coping  Skills**  will  be  validated  by  a group  of  experts  on  stress  management  for  college  students. 

This  group  will  confirm  that  the  metaphorical  implementations  have  content  and  construct  validity,  particularly  for 
Brown’s  (1992)  model  for  time  management.  They  will  also  help  to  derive  the  optimum  “healthy”  balance  of  physical, 
academic,  social,  and  psychological  needs.  An  initial  pilot  test  will  be  conducted  with  a small  group  of  college  students 
to  help  improve  interface  and  implementation  effectiveness.  The  researchers  will  interview  students  for  feedback  on 
comprehensibility  of  the  subject  matter,  clarity  or  mibiguity  of  visual  images,  accuracy,  ease  of  use,  and  relevancy; 

After  the  program  is  revised  based  on  the  evaluators’  feedback,  it  will  go  to  a main  field  test.  The  program  itself 
should  take  approximately  an  hour  to  complete.  Faculty  or  other  observers  who  supervise  the  program’s  delivery  will  be 
trained  for  inter-rater  reliability  to  assess  the  performance  of  each  student  on  completion  of  the  program.  In  addition,  the 
computer  system  will  retain  performance  logs  of  student  usage  (audit  trails).  Students  from  both  groups  may  participate 
in  pre-  and  posttests  to  measure  individual  differences  in  levels  of  anxiety  as  indicated  by  the  State-Trait  Anxiety 
Inventory.  The  researchers  may  also  test  for  visual  versus  verbal  cognitive  styles,  to  determine  if  there  is  a correlation 
with  the  ability  to  internalize  and  utilize  a new  mental  model  (Jonassen  & Grabowski,  1993).  A posttest  questionnaire 
will  again  elicit  reactions  to  comprehensibility  of  the  subject  matter,  clarity  or  ambiguity  of  visual  images,  accuracy, 
ease  of  use,  and  relevancy.  Finally,  two  weeks  later,  the  researchers  will  conduct  follow-up  interviews  of  a subset  of 
students  to  determine  if  the  construction  site  metaphor  succeeded  in  building  a lasting  mental  model.  When  these  students 
are  asked  about  friends  suffering  from  stress,  will  they  incorporate  terminology  and  analogies  from  the  construction  site 
metaphor  in  their  responses? 
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In  1986,  while  considering  the  design  of  human-computer  interfaces,  Laurel  wondered  what  the  ideal  user 
experience  might  be  and  what  sort  of  interface  might  provide  it.  Now,  ten  years  later,  a quick  market  survey  might  well 
lead  one  to  believe  that  the  answer  to  Laurel's  question  is  the  Graphical  User  Interface  (GUI).  Buttons  have  almost 
completely  replaced  function  keys.  Icons  represent  everything  from  objects  to  actions.  Once  considered  extravagant,  high 
resolution  color  graphics  and  mice  are  now  essential.  GUIs  have  become  increasingly  prevalent  across  all  operating 
platforms.  Despite  the  prevalence  of  GUIs,  however,  some  users  still  find  it  difficult  to  operate  their  computers.  It  may 
be  that  simply  employing  visual  representations  in  the  interface  is  not  enough;  to  realize  the  ideal  user  experience,  it  may 
be  necessary  to  employ  metaphors  in  interface  design. 

This  paper  chronicles  the  evolution  of  a metaphorical  graphical  user  interface  (MGUI).  This  particular  project 
has  been  a part  of  the  author's  involvement  with  the  Design  and  Development  Group  at  Lehigh  University. 

What  are  MGUIs  and  Why  Do  We  Need  Them? 

To  understand  MGUIs,  one  must  first  understand  metaphors.  The  Oxford  Encyclopedic  English  Dictionary 
(1991)  defines  metaphor  as  “the  application  of  a name  or  descriptive  term  or  phrase  to  an  object  or  action  to  which  it  is 
imaginatively  but  not  literally  applicable."  Erickson  (1990)  defined  a metaphor  as  “an  invisible  web  of  terms  and 
associations  that  underlies  the  way  we  speak  and  think  about  a concept"  (p.  66).  Mountford  (1990)  took  this  definition 
one  step  further  contending  that  "metaphors  are  powerful  verbal  and  semantic  tools  for  conveying  both  superficial  and 
deep  similarities  between  familiar  and  novel  situations"  (p.  25).  Indeed,  metaphors  frequently  appear  in  both  our  speech 
and  writing. 

A GUI's  use  of  metaphor  is,  however,  more  likely  to  be  implemented  visually.  Despite  the  above  definitions' 
failure  to  mention  visual  representations  of  an  object  or  action  to  which  it  is  imaginatively  — but  not  literally  applicable 
— a quick  stroll  through  your  neighborhood  airport  with  its  iconic  signage  should  provide  sufficient  evidence  of  the 
prevalence  of  visual  metaphors.  In  fact,  visual  metaphors  are  currently  used  in  computer  interfaces:  papers,  stacks, 
folders,  directories,  menus,  trees,  and  trash  cans  — all  generally  familiar  images  — are  manipulated  on  a desktop  to  carry 
out  the  basic  operations  of  computers.  These  metaphors  can  be  useful  and  efficient  because  they  represent  ways  of 
conceptualizing  these  computer  functions  and,  as  Erickson  (1990)  argued,  bring  along  “additional  bits  of  structure  that 
may  be  useful  later  on"  (p.  72).  And,  as  anyone  who  has  been  hesitant  before  dropping  his  or  her  file-laden  diskette  into 
Apple's  trash  can  will  tell  you,  these  lingering  bits  of  structure  do  indeed  exist.  In  fact,  because  of  this  phenomenon, 
many  designers  have  warned  that  the  inappropriate  use  of  metaphors  can  cause  confusion,  misunderstanding,  and  both 
navigational  and  conceptual  difficulties  for  your  user  (Rosendahl-Kreitman,  1990;  Semper,  1990;  Vertelney,  Arent,  and 
Lieberman,  1990). 

Assuming  for  now,  however,  that  we  can  identify  appropriate  metaphors,  the  benefits  MGUIs  might  offer 
computerized  instruction  are  twofold:  First,  because  metaphors  help  one  establish  expectations  for  how  things  should 
work  (Black,  1979),  MGUIs  might  help  a student  establish  expectations  for  how  an  instructional  software  product  should 
work.  As  is  the  case  with  basic  computer  operations  within  a MGUI  environment  where  moving  an  electronic  file  is  as 
easy  as  dragging  and  dropping  its  paper  representation  onto  a folder,  MGUI  environments  can  be  used  to  help  students 
quickly  learn  the  basic  functional  and  navigational  operations  of  an  instructional  application.  Second,  and  more 
importantly  for  instructional  design,  because  metaphors  facilitate  “understanding  and  experiencing  one  kind  of  thing  in 
terms  of  another"  (Lakoff  & Johnson,  1980,  p.  5),  MGUIs  might  also  facilitate  the  understanding  and  experiencing  of 
new  and  abstract  concepts  in  terms  of  other  more  familiar  concepts  or  experiences.  As  has  long  been  the  case  in  good 
teaching  where  educators  use  a student's  existing  knowledge  to  help  explain  new  curricular  materials  (Coker,  White,  & 
Barton,  1993),  MGUI  environments  can  be  designed  to  help  students  learn  content  by  providing  “both  superficial  and 
deep  similarities  between  familiar  and  novel  situations"  (Mountford,  1990,  p.  25). 

How  do  we  keep  the  MGUI  from  becoming,  as  Nelson  (1990)  argued,  “a  dead  weight,"  especially  when  an 
interactive  multimedia  learning  environment's  primary  purpose  is  to  teach?  Supplying  support  for  learner  inquiries  does 
not  have  to  mean  that  the  environment  must  be  boring  or  limited;  as  we  know,  there  is  certainly  an  instructional  need  to 
gain  attention  and  to  motivate.  What  this  does  mean  is  that  everything  in  the  instructional  product  must  contribute  to  the 
goal  of  helping  learners  acquire  new  knowledge  and  skills.  This  goal  easily  can  get  lost  within  a metaphor  as  its 
implementation  grows  in  complexity.  It  is  the  instructional  designer’s  task  is  to  assure  this  does  not  happen. 

One  final  note  before  we  leave  this  section.  The  defining  characteristic  of  a metaphor  is  that  it  compares  one 
thing  to  another  without  directly  stating  the  comparison  (Cates,  1993).  For  example,  a door  might  be  metaphorically 
compared  to  an  opportunity  or  a juncture  with  no  additional  explanation  as  to  why  or  how  they  are  similar.  In  this  way,  a 
metaphor  is  a statement  of  relation,  not  simply  a statement  of  fact.  Thus,  to  report  that  “a  door  is  a big  wooden  thing 
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with  a knob  on  it”  is  simply  definitional,  not  metaphorical.  Although  this  clarification  may  seem  obvious,  many  of  us 
have  discovered  that  creating  a metaphor  is  much  more  difficult  than  spotting  one.  In  fact,  this  catch-phrase  has  become  a 
warning  among  the  designers  in  Lehigh’s  Design  and  Development  Group  to  alert  us  to  GUIs  treading  dangerously  close 
to  non-metaphoricality. 


Defining  the  Content 

From  its  inception,  ‘The  Funeral  of  Edgar”  has  been  a guided  exploration  of  Edgar  Allan  Poe’s  poem,  ‘‘The 
Raven,”  aimed  at  modeling  high  school  students’  critical  and  analytical  reading  skills.  This  prod’xt  is  targeted  for  the  9th- 
grade  student  (roughly  14-15  years  old)  enrolled  in  an  English  class  with  limited  previous  exposure  to  literature  and  even 
less  experience  with  poetry.  This  is  the  kid  who  is  “psyched”  to  find  that  her  English  homework  for  the  evening  consists 
of  a two-page  poem  which  she  knows  she  can  read  in  less  than  30  seconds.  This  “MTV”  approach  to  critical  reading 
usually  ends  up  with  the  student  entirely  missing  the  poem’s  meaning  — bypassing  parenthetical  background 
information  the  textbook’s  publishers  have  so  carefully  added,  glossing  over  images  the  poet  has  carefully  chosen,  and 
failing  to  reflect  on  ways  their  own  life  experiences  are  mirrored  in  the  poem’s  themes.  Although  the  next  morning’s 
class  discussion  may  help  in  understanding  the  teacher’s  perspective,  the  student  rarely  learns  how  to  read  analytically  or 
how  to  develop  independent  ideas. 

It  appears  that  teaching  careful  and  analytical  reading  may  be  more  difficult  without  first  actually  showing  the 
student  how  it  is  done  at  the  same  time  that  the  student  reads  (Vygotsky,  1987).  Unfortunately,  given  class  sizes  and 
time  constraints,  it  is  almost  impossible  for  a teacher  to  demonstrate  critical  reading  techniques  by  guiding  each  student 
through  his  or  her  first  readings.  However,  this  kind  of  modeling  might  be  possible  from  within  a computerized  lesson 
programmed  to  guide  and  coach  each  student  actively  through  the  reading  assignment.  For  this  reason  “The  Funeral  of 
Edgar”  is  based  on  a guided  exploration  approach:  although  students  are  free  to  “roam,”  the  application  mediates  the 
student’s  experience  by  providing  content-sensitive  help,  asking  questions  at  appropriate  times,  directing  attention  to 
anomalies,  suggesting  avenues  of  further  investigation,  calling  attention  to  overlooked  information,  helping  to  organize 
the  reader’s  thoughts,  and  supporting  the  synthesis  of  new  concepts  and  schemas. 

The  Initial  Interface 

Edgar  Allan  Poe’s  ‘The  Raven”  was  chosen  for  this  application  because  of  its  reputation  as  a favorite  among 
students  within  this  age  group  and  because  the  poem’s  vocabulary  is  relatively  accessible  and  its  images  few.  “The 
Raven”  also  addresses  topics  which  describe  the  human  condition  — lost  love,  untimely  death,  human  perspective  — 
literary  subjects  which  Judy  (1980)  argued  are  appropriate  for  this  grade  level.  In  addition,  the  poem’s  supematurally 
gothic  tone  provides  an  array  of  metaphorical  opportunities  to  help  learner’s  acquire  critical  reading  skills.  Finally,  Poe’s 
“The  Philosophy  of  Composition,”  a step-by-step  discussion  which  vividly  demonstrates  Poe’s  deliberate  word  and  image 
choices  in  “The  Raven,”  provides  a unique  opportunity  for  students  to  explore  the  author’s  creative  process  while 
simultaneously  exploring  the  creation.  The  essay,  which  describes  how  poets  fabricate  their  work  from  images  selected 
for  consciously  determined  purposes,  serves  as  a “scaffold”  for  exploring  “The  Raven.” 

Okay.  Now  the  truth.  While  “The  Raven”  did,  in  fact,  fit  all  the  above  criteria,  it  would  be  less  than  candid  not 
to  admit  that  it  also  fit  nicely  into  the  bookshelf  metaphor  which  a coding  approach  to  the  content  suggested.  One  of  the 
instructional  goals  for  this  product  is  that  students  lea'  i to  use  external  resources  for  deciphering  the  multiple  meanings 
of  words  in  a poem.  “The  Philosophy  of  Composition”  nicely  rounded  out  the  bookshelf  collection:  a dictionary  for 
allowing  students  to  look  up  words,  a biography  for  allowing  students  to  read  about  Poe’s  life,  a copy  of  the  essay  so 
student’s  could  learn  about  the  creative  process,  and  the  poem  itself.  Figure  1 depicts  one  of  the  screens  from  this  early 
design. 
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FIGURE  1:  The  initial  interface. 


Although  this  interface  is  graphical,  because  it  did  not  represent  one  thing  in  terms  of  something  else  different,  it  was  in 
no  way  metaphorical.  This  type  of  design  is  a Graphical  but  Non-Metaphorical  Interface  (GNUI)  and  is  at  the  lowest  end 
of  metaphorical  interface  design  (Cates,  1996).  A door  is  a big  wooden  thing  with  a knob  on  it. 

First  Attempt  to  Make  the  Interface  Metaphorical 

As  Figure  1 shows,  in  the  project’s  early  design,  the  four  resources  were  represented  by  the  four  books  on  the 
shelf.  The  table  of  contents  and  bookshelf  screens  were  to  be  accessible  by  clicking  on  miniaturized  versions  in  the 
navigation  strip  along  the  right  side  of  the  screen.  Navigation  functions  were  represented  by  the  right/left  arrow  buttons. 

A map  of  the  entire  application  was  available  by  clicking  on  the  sign  post  at  the  lower  left.  Although  arrow  buttons  and 
sign  posts  have  nothing  to  do  with  bookshelves,  confounding  screen  devices  like  these  became  necessary  as  it  became 
apparent  how  difficult  this  bookshelf  interface  would  be  to  implement.  How  would  learners  pull  books  off  the  shelf? 
Pointing  and  clicking  seemed  unnatural.  How  would  pages  turn?  There  are  no  buttons  on  real  book’s  pages.  How  could 
the  program  be  less  passive?  With  the  exception  of  some  workbooks,  there  is  little  interactivity  with  a book.  In  this 
way,  because  of  the  inadequacy  of  the  bookshelf  metaphor,  the  initial  interface  quickly  moved  from  a GNUI  to  a Mixed 
Metaphorical  Graphical  User  Interface  (Mixed  MGUI)  — one  which  employs  an  underlying  metaphor,  but  employs  a 
variety  of  auxiliary  metaphors  that  are  confounding.  What  the  project  needed  was  a stronger  primary  metaphor. 

On  the  basis  of  Mac  Cormac  (1985),  Mountford  (1990),  and  Black’s  (1979)  discussions  of  metaphor  operation, 
one  may  divide  metaphors  into  two  classes:  primary  (or  underlying)  metaphors  and  secondary  (or  auxiliary)  metaphors.  A 
primary  metaphor  is  the  principle  or  first  metaphor  employed  and  secondary  metaphors  is  a subsequent  metaphor 
employed  by  the  product  (Cates,  1994).  The  primary  metaphor  establishes  the  general  context  into  which  the  secondary 
metaphor  must  fit. 

Poe’s  preoccupation  with  the  macabre  and  the  poem’s  overriding  death  theme  suggested  a cemetery  primary 
metaphor  in  which  learners  might  explore  a variety  of  objects  and  characters.  A main  screen  filled  with  objects  and 
characters  upon  which  the  learner  clicked  might  eliminate  all  need  for  confounding  navigational  devices.  In  addition,  if  the 
metaphor  were  actually  Poe's  fictional  funeral,  it  would  be  possible  to  have  his  ghost  deliver  excerpts  from  ‘The 
Philosophy  of  Composition”  rather  than  to  make  students  trudge  through  its  somewhat  heavy-handed  prose.  Having 
Poe’s  spirit  “speak”  directly  to  the  students  from  the  grave  about  his  c^ative  process  while  they  read  the  poem  might 
help  them  understand  the  process  better.  Twin  brothers  could  eulogize  the  poet  by  having  one  read  from  the  poem  while 
the  other  added  personal  anecdotes  and  interpretations.  Figure  2 shows  the  main  screen  from  this  first  MGUI  attempt. 
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FIGURE  2:  The  main  screen  from  the  first  MGUI  attempt. 


This  screen  depicted  several  objects  and  characters  upon  which  learners*  clicked  to  “zoom  in”  and  learn  pieces  of 
background  information  about  the  poem.  For  example,  when  learners  clicked  on  Poe’s  op^n  grave  his  ghost  appeared  and 
briefly  outlined  the  “Philosophy  of  Composition.”  When  learners  clicked  on  the  caretaker,  always  available  to  provide 
help,  he  gave  dynamic  suggestions  on  where  to  click  next.  When  learners  clicked  on  a tombstone  marked  “Edgar”  their 
perspective  moved  in  close  enough  to  see  its  engraving  of  Poe,  his  dates  of  birth  and  death,  and  other  brief  details  about 
his  life.  When  learners  clicked  on  a second  tombstone  next  to  Poe’s  marked  “Virginia,”  his  wife  described  their  love  and 
her  untimely  death.  Other  objects  included  a statue  of  Pallas  Athena  (Greek  goddess  of  wisdom  who  briefly  described  her 
story  and  Poe’s  fascination  with  “forgotten  lore”),  a raven  in  a tree  (who  said  only  “Nevermore”),  and  a suspicious- 
looking  character  peering  from  behind  a bush  (one  of  Poe’s  critics  who  discussed  his  views). 

After  the  learner  had  explored  all  these  objects,  the  twin  mourners  arrived  to  do  the  eulogy.  While  they  spoke  in 
the  “foreground,”  Poe’s  ghost  occasionally  floated  through  the  background  dropping  roses  (taken  from  the  fact  that 
someone  leaves  a rose  at  Poe’s  grave  in  Baltimore  every  year  on  the  anniversary  cf  Poe’s  birthday).  When  the  learner 
clicked  on  these  roses,  the  caretaker  was  replaced  by  Poe’s  ghost  who  provided  excerpts  from  “The  Philosophy  of 
Composition”  pertinent  to  the  stanza  being  covered  at  that  time  by  the  twins  (see  Figure  3). 
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FIGURE  3:  The  “eulogy”  screen. 
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The  cemetery  metaphor  lent  itself  to  producing  a lesson  that  was  likely  to  be  much  more  interesting  and 
infinitely  more  entertaining  than  the  bookshelf  version  would  have  been.  The  primary  metaphor  had  established  the 
general  context  in*c  which  all  of  the  secondary  metaphors  fit.  All  of  the  images  used  were  consonant  with  the  cemetery 
theme  and  enhanced  the  metaphor’s  believability  and  efficacy.  However,  the  application  was  still  primarily  just  a “point- 
and-click”  exercise.  Learners  using  this  product  were  barely  more  involved  in  this  interface  than  they  would  have  been 
with  the  bookshelf.  Consequently,  this  design  fell  into  the  third  group  on  the  metaphorical  interface  design  continuum,  a 
Thematic  Metaphorical  Graphical  User  Interface  (Thematic  MGUI  — Cates,  1996).  A door  is  a hinged,  sliding,  or 
revolving  barrier  for  closing  and  opening  an  entrance  — with  a knob  on  it.  Back  to  the  drawing  board. 

Implementing  an  Immersive  MGUI 

Although  thematic  GUIs  may  enhance  believability  and  predictability  of  the  application,  they  tend  to  be 
somewhat  flat.  In  this  case,  all  interaction  with  the  program  had  become  simple  back-and-forth  navigation  zooming  in 
and  away  from  objects  on  the  screen.  Moving  to  the  most  metaphorical  of  the  graphical  user  interfaces,  the  Immersive 
MGUI,  requires  adding  a third  dimension  — one  that  creates  an  interactive  environment  for  the  user.  Thus,  the  first  step 
toward  implementing  an  immersive  MGUI  was  to  create  a scenario  in  which  the  application  could  become  more 
interactive  — by  requiring  learners  to  take  on  the  role  of  the  cemetery’s  caretaker  rather  than  be  just  passive  observers  as 
had  been  the  case  in  the  application’s  thematic  GUI  version.  In  order  to  get  the  learners  involved  in  their  immersive  role 
as  quickly  as  possible,  the  application  opens  with  an  angry  funeral  director  asking  where  the  learner  has  been  while 
preparations  for  the  Poe  funeral  should  have  been  made. 

In  addition  to  changes  in  the  scenario,  it  was  necessary  to  make  some  other  general  changes  to  the  interface  in 
order  to  make  it  more  immersive.  For  example,  to  draw  learners  into  the  poem’s  supernaturally  gothic  tone  and  increase 
the  interface’s  three-dimensional  feel,  grim  photographic  graphics  were  chosen  to  replace  the  more  cartoon-like  graphics 
that  had  been  used  in  the  thematic  GUI  version.  Figure  4 shows  the  changes  that  have  been  made  to  the  main  screen. 


FIGURE  4:  Main  screen  from  which  all  exploration  begins. 

In  addition  to  setting  up  the  scenario  for  increased  learner  involvement,  the  brooding  funeral  director  enhances  the 
immersive  MGUI’s  foreboding  mood  by  replacing  the  smiling  caretaker  that  had  been  a large  part  of  the  earlier  version. 

Three-dimensional  objects  and  enhanced  manipulatability  are  key  components  of  immersive  MGUIs.  To  make 
“The  Funeral  of  Edgar”  interface  more  interactive,  manipulatable  objects  such  as  an  obituary  page,  flowers,  and  charcoal 
(for  stone  rubbings)  were  added.  Instead  of  simply  pointing  and  clicking  at  these  objects,  as  had  been  the  case  with  the 
thematic  GUI,  objects  within  the  new,  more  immersive  GUI  are  dragged  and  dropped.  Figure  5 shows  a screen  in  which 
users  learn  more  about  the  poet  by  reading  the  cards  enclosed  with  flowers  left  for  the  deceased. 
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FIGURE  5:  Card  reading  screen. 

Rather  than  simply  pointing  and  clicking  on  the  flowers  to  get  a “blown  up”  version  of  the  card,  learners  must  actually 
“pull”  the  card  out  of  the  envelope  by  dragging  it  out  and  dropping  it.  Changes  in  the  cursor  from  a “pointer”  hand,  to  a 
“grabbing”  hand,  to  a “go  back”  hand  enhance  the  realism  of  these  manipulations  and  make  it  possible  to  navigate  while 
minimizing  cumbersome  and  intrusive  mouse  clicks  (see  figures  6-8). 


FIGURE  6:  Pointer  cursor 
indicates  manipulatable 
object. 


FIGURE  7:  Grabbing  hand 
indicates  draggable  object. 


FIGURE  8:  “Go  back”  hand 
provides  means  to  restore 
full  screen. 


To  establish  the  feel  of  a larger  “virtual”  world  and  to  eliminate  the  need  for  an  extensive  navigation  scheme,  the 
application  was  programmed  so  that  the  screen  is  literally  a small  window  on  an  interface  that  spans  three  screen  widths. 
When  learners  click  on  the  left  or  right  side  of  the  screen,  the  program  pans  left  and  right  across  the  interface.  Small  free- 
standing directional  signs,  like  the  ones  which  might  be  found  in  a cemetery  to  direct  visitors,  are  positioned  in  the  lower 
left  and  right  comers  of  the  screen  to  indicate  this  functionality. 

Cates  (1994)  suggested  that  “properties,  operations,  phrases,  images,  and  types  associated  with  the  primary 
metaphor  may  provide  the  imaginal  and  semantic  links”  to  related,  secondary  metaphors  (p.  3).  The  primary  funeral 
metaphor  lends  itself  to  many  secondary  metaphors:  flowers,  mourners,  tombstones,  graves,  epitaphs,  caskets,  ghosts, 
eulogies,  and  seances.  These  devices  are  used  to  provide  content-  and  context-sensitive  help,  introduce  historical 
background,  produce  Poe’s  poetic  philosophy,  and  deliver  the  poem  itself.  As  before,  students  gather  information 
pertinent  to  the  poem  from  the  various  characters  and  objects  by  clicking  on  them.  As  learners  explore  each  of  these 
objects,  time  passes  on  the  sundial  eventually  triggering  events  like  the  arrival  of  the  casket  and  flowers,  the  burial 
ceremony,  and  the  seance.  Before  exiting  the  program  learners  pass  through  a “field”  of  grave  markers  where  they  see  that 
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the  program  has  “marked”  their  place  before  exiting  with  an  epitaph  like  “Here  lies  <student’s  name>. . . we  look  forward 
to  his  next  coming.” 

Provided  one  does  a good  job  selecting  secondary  metaphors,  choosing  a primary  metaphor  might  appear  at  first 
to  be  a somewhat  arbitrary  decision.  In  the  thematic  MGUI  version  of  “The  Funeral  of  Edgar,”  the  primary  cemetery 
metaphor  was  generally  location-based.  Although  location-based  metaphors  may  lend  themselves  to  teaching  static 
materials  which  the  learner  needs  to  memorize,  the  cemetery  metaphor  left  few  alternatives  for  segmenting  the  complex 
content  that  needed  to  be  covered.  Consequently,  screens  like  the  “eulogy  screen”  depicted  in  Figure  3 became  overloaded. 
In  contrast,  the  new  version  of  the  lesson  relies  or.  a funeral  as  its  primary  metaphor;  a subtle  shift  that  proved  to  be  a 
major  factor  toward  making  its  MGUI  more  immersive.  This  change  allowed  the  lesson’s  content  to  be  metaphorically 
divided  by  time:  in  the  period  before  the  funeral  learners  interact  with  screen  objects  to  learn  background  information 
about  the  poet  and  about  images  used  in  the  poem;  during  the  funeral  learners  interact  with  characters  to  learn  more  about 
how  differently  individuals  interpret  events;  after  the  funeral  students  learn  that  a poem’s  text  includes  words  that  have 
multiple  “layers”  of  significance  to  its  overall  theme. 

Concluding  Remarks 

Thus,  “The  Funeral  of  Edgar”  evolved  slowly  from  a GNUI  to  an  immersive  MGUI  in  the  hope  that  its  audience 
will  feel  a greater  sense  of  believability  and  involvement,  which  might  lead  to  better  comprehension  (Cates,  1996). 
Granted,  many  14-  to  15-year-old  students  will  not  have  personal  experience  with  funerals.  However,  their  knowledge  of  a 
graveyard’s  visual,  functional,  and  emotive  properties  (from  television,  movies,  and  books)  should  be  sufficient  for  the 
primary  and  secondary  metaphors  to  operate.  In  addition,  these  learners  will  be  familiar  with  asking  others  for 
information  (as  opposed  to  consulting  reference  materials)  and  with  various  computer  and  video  games  that  utilize  similar 
exploration  formats. 

So,  a door  is  an  opportunity  and  a gate  is  something  you  use  to  get  into  a cemetery  — but  don’t  expect  that  to 
be  all  that  there  is  to  it. 
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Abstract 


The  purpose  of  this  study  was  to  determine  whether  integrating  cooperative  learning  strategies  with  ILS- 
delivered  instruction  in  mathematics  produced  academic  and  attitudinal  gains  in  students  and,  if  so,  to  discern  if  this 
strategy  was  more  effective  for  students  with  high  or  low  academic  ability  levels.  In  order  to  gauge  the  efficacy  of  the 
intervention,  achievement  and  attitudinal  data  were  collected  for  all  fifth  grade  students  in  the  selected  school  prior  to  the 
experimental  treatment  and  at  the  end  of  the  treatment  period.  Results  of  this  study  revealed  that  students  using  an  ILS 
for  mathematics  instruction  performed  better  on  standardized  tests  when  they  completed  the  computer  activities  in 
cooperative  groups.  In  addition,  attitudes  and  behaviors  were  more  positive  towards  math  and  the  computer  math 
activities  when  the  students  worked  in  cooperative  groups. 

In  recent  years,  many  schools  have  turned  to  Integrated  Learning  Systems  (ILSs)  to  facilitate  instruction  and 
assist  with  raising  state  standardized  test  scores  (Becker,  1994).  Typically,  these  ILSs  are  utilized  in  a computer  lab, 
where  teachers  brings  their  classes  into  the  lab  and  students  work  individually  on  computer-based  instructional  lessons  for 
an  allotted  time  period.  In  this  model  of  ILS  delivery,  students  are  generally  isolated  from  one  another,  and  are 
discouraged  from  sharing  information  or  providing  assistance  to  other  students.  Although  using  ILSs  in  this  manner 
may  fit  well  with  the  initial  design  and  purpose  of  ILSs  (i.e.  instruction  tailored  to  individual  needs),  it  has  also  created 
problems  such  as  increasing  anxiety,  hostility,  and  boredom  in  students  (Lepper,  1985). 

Through  the  integration  of  cooperative  learning  with  ILS  delivery,  some  of  these  problems  may  be  alleviated. 
This  study  examined  achievement  and  behavior  differences  between  students  completing  ILS  activities  in  a traditional, 
individualized  format,  and  students  completing  the  same  activities  in  cooperative  learning  groups  in  order  to  determine  if 
a cooperative  learning  model  could  be  used  effectively  with  students  completing  mathematics  activities  in  a lab-based 
ILS. 


THEORETICAL  BACKGROUND 

Cooperative  Learning 

Cooperative  learning  as  an  instructional  strategy  for  students  of  varying  achievement  levels  has  been  widely 
researched  over  the  past  twenty  years.  In  a typical  cooperative  learning  situation,  students  are  grouped  together  in  some 
manner  and  asked  to  work  together  to  complete  a task,  such  as  to  master  ;ome  instructional  material,  develop  a report  on 
certain  subject  matter,  or  complete  a creative  project  (Lloyd,  Crowley,  Kohler,  & Strain,  1988).  Rewards  for  the 
individual  students  within  the  group  are  generally  based  on  the  accomplishments  of  the  group  as  a whole.  Proponents  of 
cooperative  learning  argue  that  having  groups  of  students  work  together  towards  a common  goal  with  shared  rewards  will 
lead  to  both  academic  and  social  gains  for  all  students  participating  in  the  lessons  (Johnson  & Johnson,  1982;  Mesch, 
Lew,  Johnson,  & Johnson,  1986). 

Researchers  have  proposed  a variety  of  cooperative  learning  models  in  various  instructional  setting.  However, 
Slavin  (1985)  has  noted  that  while  different  cooperative  learning  models  may  vary  in  implementation,  they  are  generally 
utilizing  alternative  methods  for  ensuring  that  the  key  elements  of  cooperative  learning  are  integrated  into  the 
instructional  task.  These  key  elements  of  cooperative  learning,  as  outlined  by  leaders  in  cooperative  learning  research 
such  as  Johnson  and  Johnson  (1987),  and  Slavin  (1990,  1980),  include: 

(1)  Positive  interdependence.  Students  must  believe  that  each  one  of  them  has  a key  role  in  the  group,  and  that 
the  task  to  be  accomplished  cannot  be  completed  without  the  actions  of  each  member. 

(2)  Individual  accountability.  Each  student  within  a group  must  be  held  accountable  for  mastery  of  the 
instruction  presented  to  the  group. 

(3)  Group  rewards.  There  must  be  sufficient  incentives  for  the  group  to  remain  together.  The  accomplishments 
of  the  group  as  a whole  should  be  rewarded  just  as  the  accomplishments  of  each  individual  are  rewarded. 

(4)  Group  training.  Students  cannot  be  placed  together  in  a group  situation  and  be  expected  to  cooperate.  As 
Johnson  and  Johnson  (1987)  state,  "students  must  be  taught  the  social  skills  needed  for  collaboration,  and  they  must  be 
motivated  to  use  them"  (p.  13). 

Cooperative  Learning’ sJBffects  on  Achievement 

Research  examining  the  academic  impact  of  cooperative  learning  has  provided  mostly  positive  results.  Slavin 
(1983)  examined  46  studies  in  which  cooperative  learning  groups  were  compared  with  individual  instruction.  Of  those  46 
studies,  he  found  that  29  reported  a significant  increase  in  achievement  levels  for  students  participating  in  cooperative 
learning  groups,  with  another  15  showing  no  significant  differences.  Slavin  further  noted  that  the  studies  which  did  not 
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show  learning  gains  for  students  in  cooperative  learning  groups  did  not  incorporate  one  or  more  of  the  important 
components  of  cooperative  learning  outlined  above.  In  a second  meta-analysis,  Slavin  (1987)  analyzed  studies  which 
compared  the  effects  of  cooperative  learning  on  achievement  with  those  of  students  in  individual  learning  situations.  Of 
the  38  studies  reviewed,  Slavin  found  that  33  reported  significant  increases  in  academic  achievement  for  students 
participating  in  cooperative  learning  situations. 

Research  has  also  examined  the  effectiveness  of  cooperative  learning  with  students  of  different  academic  ability 
levels.  Many  proponents  of  cooperative  learning  believe  that,  when  implemented  properly,  cooperative  learning  is 
effective  for  students  of  all  academic  ability  levels.  However,  some  researchers  argue  that  cooperative  learning  is  not  as 
effective  with  students  of  low-ability  levels.  For  example,  in  a qualitative  study  by  King  (1993),  third-grade  students 
received  mathematics  instruction  in  cooperative  groups  consisting  of  two  students  classified  as  being  high-achieving  and 
two  students  classified  as  being  low-achieving.  Observations  of  the  students  working  in  the  cooperative  groups  revealed 
that  students  classified  as  low-achieving  tended  to  make  minimal  contributions  to  the  group  work  and  took  on  passive 
roles  within  the  group,  whereas  the  students  classified  as  high  achieving  completed  the  majority  of  the  group  work 
without  input  from  the  other  members  of  the  group.  This  research  also  noted  that  "...low  achievers  were  not  learning  to 
any  significant  degree.  Lows'  self-perceived  lack  of  progress  in  small  groups  may  indicate  that  their  developing  of 
mathematical  understanding  and  skills  is  not  enhanced  by  small-group  work."  (King,  1993,  p.  413). 

Other  research,  however,  has  shown  that  low-ability  students  benefit  from  cooperative  learning  strategies.  In  a 
study  by  Simsek  and  Hooper  (1992),  30  fifth  and  sixth  grade  students  worked  either  independently  or  cooperatively  on  a 
videodisc  lesson  about  whales.  The  cooperative  groups  were  composed  of  either  one  student  classified  as  "low  ability" 
and  two  students  classified  as  "high  ability,"  or  two  students  classified  as  "low  ability"  and  one  student  classified  as  "high 
ability."  Posttest  results  revealed  that  students  participating  in  cooperative  groups,  regardless  of  ability  level,  scored 
significantly  higher.  Similar  positive  effects  of  cooperative  learning  with  students  of  varying  ability  levels  were 
discussed  by  Hooper  (1992),  Watson  and  Rangel  (1989),  Slavin  (1991),  and  Johnson  and  Johnson  (1992). 

Cooperative  Learning's  Effects  on  Student  Attitudes  and  Social  Behaviors 

Research  shows  that  cooperative  learning  can  have  a positive  impact  on  students'  social  behaviors  and  attitudes 
towards  instructional  content.  In  a study  by  Farivar  (1992),  seventh-grade  students  worked  on  mathematics  assignments 
in  cooperative  groups.  Results  of  the  study  indicated  improved  attitudes  towards  both  cooperative  learning  activities  and 
mathematics  in  general.  Good,  Reys,  Grouws,  and  Mulryan  (1989-1990)  found  that  cooperative  group  work  in 
mathematics  increased  students'  motivation  and  enthusiasm  towards  the  subject  matter.  Similar  findings  were  reported  by 
Davidson  and  Kroll  (1991)  and  Slavin  (1985). 

In  terms  of  effects  on  social  behaviors,  Mesch,  Lew,  Johnson,  and  Johnson  (1986)  placed  students  in 
cooperative  learning  situations  and  provided  them  with  training  on  effectively  interacting  in  those  situations.  As  a result 
of  the  training  and  cooperative  group  experiences,  these  researchers  found  that  students  who  tended  to  be  isolated  and 
withdrawn  interacted  significantly  more  with  their  peers  both  within  and  outside  the  cooperative  learning  activities.  In  a 
meta-analysis  of  five  studies  dealing  with  cooperative  learning,  Lloyd,  Crowley,  Kohler,  and  Strain  (1988)  concluded  that 
cooperative  learning  had  significant  positive  effects,  "particularly  on  social  behavior,  in  comparison  to  competitive  and 
individualistic  procedures."  (p.  43).  Several  studies  have  found  that  cooperative  learning  improves  relationships  between 
disabled  and  ncn-disabled  peers  (Armstrong,  Johnson,  & Balow,  1981;  Madden  & Slavin,  1983;  Yager,  Johnson, 

Johnson,  & Snider,  1985),  and  between  students  from  different  ethnic  and  racial  backgrounds  (Slavin,  1983).  Research 
has  also  shown  cooperative  learning  to  decrease  behavior  difficulties  such  as  talking  out  and  not  paying  attention  in  the 
classroom  (Greenwood,  Carta,  & Hall,  1988;  Mulryan,  1995;  Tyrrell,  1990). 

Integrating  Cooperative  Learning  with  Computer-  Assisted-Instruction  and  ILSs 

As  interest  in  applying  cooperative  learning  in  a wider  variety  of  instructional  settings  has  grown,  there  has 
been  an  increase  in  research  investigating  integration  of  cooperative  learning  with  other  instructional  tools.  One  such 
combination  is  cooperative  learning  and  computer-assisted-instruction  (CAI).  Some  have  argued  that,  by  providing 
students  with  peer  support  and  assistance  when  utilizing  traditionally  individualized  tools  such  as  CAI,  cooperative 
learning  can  increase  the  effectiveness  those  tools  have  on  student  achievement  (Becker,  1992).  For  example,  Mevarech, 
Stern,  and  Levita  (1987)  placed  junior  high  school  students  into  cooperative  learning  groups  and  asked  the  groups  to 
complete  language  arts  CAI  lessons.  Results  indicated  that  students  who  worked  on  the  CAI  lessons  in  cooperative 
groups  demonstrated  significantly  greater  academic  gains  than  their  peers  working  on  the  CAI  lessons  individually. 
Similar  positive  results  when  combining  cooperative  learning  and  CAI  were  found  by  Hooper  and  Hannafin  (1991),  and 
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Hooper,  Temiyakarn,  and  Williams  (1993).  In  these  studies,  peer  interactions,  positive  student  attitudes  towards  the 
group  activities,  and  the  construction  of  a supportive  learning  environment  were  all  determined  to  be  factors  which 
contributed  to  positive  academic  gains  for  students. 

In  the  past  few  years,  CAI  has  evolved  into  powerful  and  costly  systems,  such  as  ILSs  (Wiburg,  1995).  An  ILS 
is  generally  implemented  in  a computer  lab  of  15  to  30  computers  linked  to  a central  fileserver.  The  fileserver  contains 
instructional  software  which  contains  entire  curricula  in  reading,  writing,  mathematics,  science,  or  other  disciplines,  over 
several  grade  levels  (Bender,  1991;  Robinson,  1991).  This  software  is  individually  distributed  to  each  computer  in  the 
computer  lab,  so  that  each  student  using  the  lab  can  work  on  any  lesson  at  any  given  time.  With  this  system,  teachers 
(or  students)  can  simply  select  the  lessons  they  wish  to  use  by  choosing  from  a menu,  or  they  can  allow  the  computer  to 
determine  the  appropriate  content  and  difficulty  level  for  particular  students.  In  the  latter  case,  the  ILS  selects  and  assigns 
lef,sons  which  either  introduce  new  concepts  or  provide  remediation  of  more  difficult  concepts  according  to  the  needs  of 
f;ach  student.  The  ILS  can  also  maintain  detailed  records  of  each  student’s  progress  and  performance  which  the  teacher  can 
access  at  any  time  for  assessment  and  evaluation  purposes. 

The  research  dealing  with  the  academic  effectiveness  of  combining  cooperative  learning  with  ILS-delivered 
instruction  is  somewhat  limited.  Mevarech  (1994)  investigated  the  impact  of  using  a cooperative  versus  individual  model 
when  using  an  ILS  system  developed  in  Israel.  Results  of  this  study  indicated  that  students  working  in  cooperative 
groups  performed  significantly  better  on  tests  of  both  mathematics  basic  skills  and  higher-level  concepts  than  their  peers 
working  individually. 

Knowledge  about  the  effectiveness  of  integrating  ILSs  and  cooperative  learning  could  prove  invaluable  to 
teachers  and  administrators  who  are  struggling  to  incorporate  ILS  instruction  into  their  curriculum.  The  large  cost  of 
establishing  and  maintaining  such  systems,  coupled  with  limited  resources,  force  many  school  districts  to  target  specific 
students  to  use  ILS  resources  (Becker,  1994).  This  becomes  necessary  when  schools  do  not  have  the  facilities  to  serve  all 
students.  If  cooperative  learning  in  an  ILS  setting  were  shown  to  be  effective,  not  only  will  students  benefit  from  an 
academic  and  social  learning  perspective,  but  school  districts  will  be  able  to  utilize  their  ILS  resources  more  efficiently 
by  serving  more  students. 

The  purpose  of  this  study  was  to  determine  whether  integrating  cooperative  learning  strategies  with  ILS- 
delivered  instruction  produced  positive  academic  and  attitudinal  gains  in  students.  In  addition,  this  study  examined  the 
effectiveness  of  combining  ILS  and  cooperative  learning  with  students  of  different  academic  ability  levels.  Student 
achievement  test  data,  results  from  an  attitudinal  questionnaire,  and  researcher  observations  of  student  behavior  were  used 
to  determine  significant  differences  between  the  treatment  groups. 

METHOD 

Subjects  and  Setting 

The  subjects  were  65  fifth  grade  students  in  one  selected  elementary  school  located  in  a small  city  in  the  upper 
midwest.  The  school  selected  for  the  study  was  the  second -largest  of  the  three  elementary  schools  in  the  school  district, 
with  an  overall  enrollment  of  approximately  520  students.  The  teacher/  student  ratio  for  the  selected  school  was 
approximately  1 :20,  with  an  average  class  size  of  25  students. 

The  children  enrolled  in  the  school  were  generally  from  households  with  low  to  lower-middle  socioeconomic 
status,  and  the  average  income  of  families  served  by  the  school  was  approximately  $12,000.  As  a result  of  the  low 
socioeconomic  levels  in  the  community,  approximately  43%  of  the  students  served  by  this  school  were  eligible  for  free 
or  reduced  lunches  under  federal  guidelines.  The  school  also  served  children  whose  parents  were  military  personnel,  which 
accounted  for  approximately  20%  of  the  total  student  population.  The  ethnic  distribution  of  the  students  was 
approximately  60%  Caucasian,  30%  African-American,  and  10%  from  other  ethnic  groups. 

Initially,  71  students  were  eligible  to  participate  in  the  study  by  virtue  of  their  enrollment  in  one  of  the  three 
fifth-grade  classrooms  at  the  school.  However,  two  students  left  the  district  during  the  second  week  of  school,  two 
students  left  school  immediately  after  completing  the  pre-tests,  and  parental  consent  was  not  received  from  two  other 
students.  One  new  student  enrolled  in  the  school  during  the  fifth  week  of  the  study,  but  was  not  included  in  data 
collection.  Thus,  a total  of  65  students  participated  in  the  study.  The  ethnic  breakdown  of  the  students  participating  was 
70%  Caucasian,  25%  African-American,  and  5%  Asian- American.  Thirty-seven  percent  of  the  students  were  female,  and 
63%  were  male. 

Students  participating  in  this  study  had  already  used  the  ILS  labs  and  the  computerized  mathematics  curriculum 
for  a year,  so  training  on  use  of  the  computer  was  not  necessary.  Any  students  new  to  the  district  received  an  orientation 
to  the  ILS  lab  as  mandated  by  school  computer-use  policies. 
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Design  and  Materials 

This  study  used  a 2 X 2 (treatment  group  X ability  level)  posttest-only  design  to  determine  achievement 
differences.  All  subjects  were  blocked  by  ability  based  on  pretest  scores.  The  dependent  variable  was  mathematics 
achievement.  This  variable  was  operationally  defined  as  the  achievement  test  post-test  scores. 

Since  intact  classes  were  used  in  assigning  students  to  treatments,  the  difference  in  group  pretest  scores  could 
not  be  fully  accounted  for  experimentally.  The  pretest  mean  of  students  in  the  control  group  (M  = 45.95)  was  somewhat 
higher  than  the  mean  of  the  students  in  the  experimental  group  (M.  = 45.07).  This  difference  in  pretest  means  was  not 
significant,  F(l,64)  = 0.88,  p =.352,  however,  owing  to  the  importance  of  student  ability  in  the  present  study  posttest 
data  were  analyzed  using  an  analysis  of  covariance  (ANCOVA)  with  student  pretest  scores  as  the  covariate.  The 
ANCOVA  procedure  is  recommended  by  Taylor  and  Innocenti  (1993)  to  increase  statistical  power. 

To  determine  attitudinal  differences  between  the  treatment  groups,  separate  chi-squares  were  calculated  using  the  • 
student  responses  obtained  for  each  of  the  attitudinal  questions  on  the  student  exit  questionnaire. 

Testing  Instrument  - The  California  Achievement  Test,  Fifth  Edition  (CAT/5),  was  used  as  both  the  pre-  and 
post-test  instrument  for  mathematics  achievement.  For  this  study,  students  completed  parallel  forms  of  Level  15  of  the 
entire  mathematics  battery  (Form  A for  the  pre-test  and  Form  B for  the  post-test).  Level  15  is  the  grade  5 level  of  the 
test  (grade  range  4.6  - 6.2).  Both  versions  of  the  test  consisted  of  94  questions,  44  of  which  pertained  to  math 
computation  and  50  to  math  concepts  and  applications.  The  mathematics  computation  portion  of  the  test  covered 
traditional  computation  operations  such  as  whole  numbers,  decimals,  fractions,  integers,  percents,  algebraic  expressions, 
estimation  and  using  hand-held  calculators.  The  mathematics  concepts  and  applications  portion  focused  on  problem 
solving,  which  includes  the  "...broad  range  of  strategies  and  critical-thinking  skills  students  use  in  confronting  problems 
based  in  any  area  of  mathematics."  (CTB/Macmillan/McGraw-Hill,  1992,  p.9). 

Level  15  of  the  mathematics  section  of  the  test  has  a standardized  mean  of  58.38,  a standard  deviation  of  18.00, 
and  a standard  error  of  measurement  of  4.01.  Split- half  reliability  of  the  test,  as  measured  by  the  Kuder-Richardson 
Formula  20  measure  of  internal  consistency  was  0.95  (CTB/Macmillan/McGraw-Hill,  1992). 

Curriculum  - The  Jostens  mathematics  curriculum  was  used  as  the  instructional  content.  This  curriculum 
covered  the  entire  range  of  the  5th  grade  curricular  objectives  for  mathematics  for  the  state  of  Michigan,  as  outlined  by 
the  Michigan  Essential  Goals  and  Objectives  for  Mathematics  (Jostens  Learning  Corporation,  1990).  The  mathematical 
concepts  covered  in  these  objectives  for  grade  five  include  whole  numbers  and  numeration;  fractions,  decimals,  ratios,  and 
percents;  measurement;  geometry;  statistics  and  probability;  algebraic  ideas;  problem  solving  and  logical  reasoning;  and 
calculators.  These  concepts  also  corresponded  to  the  objectives  covered  on  the  California  Achievement  Test  (Jostens 
Learning  Corporation,  1993). 

Delivery  - The  instruction  was  delivered  via  a networked  ILS  lab,  with  30  computer  stations  in  the  lab.  With 
this  delivery  method;  each  student  (or  pair  of  students)  received  mathematics  lessons  as  determined  by  their  performance 
and  progress. 

Training  Documents  - Training  materials  and  activities  were  developed  to  assist  students  with  establishing  their 
cooperative  groups,  and  to  help  students  understand  their  roles  and  responsibilities  as  members  of  groups.  There  were 
two  separate  cooperative  activities  used  to  provide  training  in  cooperative  learning  strategies,  one  domain-general  and  the 
other  domain-specific.  The  domain-general  activity  provided  students  with  an  opportunity  to  work  together  to  complete  a 
task  that  was  not  related  to  any  particular  content  area.  The  second  activity  specifically  dealt  with  mathematics  concepts. 

Student  Exit  Questionnaire  - A student  exit  questionnaire  was  developed  by  the  researcher  and  administered  to 
participating  students  during  the  last  week  of  the  study.  There  were  seven  questions  on  the  survey,  asking  students  about 
their  attitudes  regarding  math,  the  computer  math  lessons,  and  group  activities.  The  survey  was  reviewed  and  approved  by 
a group  of  education  experts  prior  to  administering  the  survey  to  students. 

Procedure 

At  the  beginning  of  the  third  week  of  school,  the  classroom  teachers  administered  the  pre-test  to  the  students 
participating  in  the  study.  The  test  took  two  days  to  complete.  Students  completed  the  first  part  of  the  pre-test 
(computation)  on  the  first  day  and  the  second  part  of  the  pre-test  (concepts  and  applications)  on  the  second  day.  Each 
student  completed  the  pre-test  individually. 

Students  scoring  below  the  group  median  for  the  pretest  were  classified  “low-achieving,”  students  scoring  at  or 
above  the  median  were  classified  as  “high-achieving.”  This  provided  32  students  in  the  low-achieving  group  and  33 
students  in  the  high-achieving  group.  Students  in  one  of  the  three  fifth  grade  classes  were  randomly  assigned  to  the 
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individual  learning  (control)  group,  and  students  in  the  other  two  classes  were  assigned  to  the  cooperative  (experimental) 
group. 

For  the  two  experimental  classes,  each  student  was  randomly  paired  with  another  student  in  their  class.  Thus, 
since  there  were  44  students  in  the  experimental  classes,  a total  of  22  cooperative  groups  were  formed.  These  student 
pairs  then  received  training  on  cooperative  learning  and  group  interaction.  All  student  teams  completed  the  training  prior 
to  starting  their  work  on  the  computers.  The  21  students  in  the  control  class  did  not  receive  any  training. 

At  the  beginning  of  the  fourth  week  of  school,  all  students  participating  in  the  study  started  working  on  the 
computer  lessons.  Students  in  the  experimental  group  worked  on  the  lessons  with  their  partners,  while  students  in  the 
control  group  worked  individually  on  the  lessons.  Students  worked  on  the  computer  math  lessons  one  time  a week  for 
50  minutes  each  session.  Students  in  the  experimental  group  worked  with  their  partners  for  the  entire  50  minutes.  The 
classroom  teacher  and  lab  assistant  were  present  in  the  lab  during  each  session. 

Students  in  both  the  experimental  group  and  the  control  group  were  observed  by  the  researcher  during  their 
mathematics  lab  time  each  week  throughout  the  study.  The  purpose  of  the  observations  was  to  collect  anecdotal  and 
qualitative  data  regarding  student  actions  and  work  habits  in  the  computer  lab.  The  researcher  observed  for  30  of  the  50 
minutes  of  each  session,  allowing  the  first  10  minutes  for  students  to  move  to  their  assigned  seats,  log  in,  and  begin 
working,  and  the  last  10  minutes  for  students  to  print  their  results  and  save  their  work. 

The  researcher  used  a pre-developed  observation  form  to  record  any  student  comments  or  questions  towards  the 
teacher  or  lab  assistant,  student  comments  to  themselves  or  their  partners,  and  the  frequency  with  which  each  student  or 
group  requested  assistance  from  the  teacher  or  lab  assistant.  In  collecting  these  data,  the  researcher  observed  from  an 
unobtrusive  location  in  the  computer  lab.  The  researcher  never  interacted  with  the  students  during  observations. 
Occasionally,  the  researcher  would  ask  the  teacher  or  lab  assistant  for  clarification  on  interactions  that  took  place.  The 
researcher  met  with  the  computer  lab  assistant  at  the  end  of  each  week  to  discuss  potential  interaction  problems  with  the 
student  pairs,  and  to  obtain  feedback  regarding  the  performance  of  the  students  in  the  lab. 

All  students  participating  in  the  study  were  also  asked  to  complete  attitudinal  questionnaires  during  the  final 
week  of  the  study.  The  questionnaire  asked  students  questions  dealing  with  their  attitudes  towards  mathematics,  the 
computer  math  lessons,  and  working  in  cooperative  groups.  The  questionnaires  were  administered  to  the  students  in  the 
computer  lab  by  the  lab  assistant. 

Students  in  the  study  worked  on  computerized  math  lessons  once  a week  for  a period  of  1 1 weeks.  At  the  end  of 
the  treatment  period,  the  students  participating  in  the  study  were  given  a post-test  using  the  mathematics  portion  of  form 
B of  the  CAT/5.  The  protocol  for  administration  of  the  post-test  was  identical  to  that  of  the  pre-test.  As  with  the  pre- 
test, each  student  completed  the  post-test  individually. 


RESULTS 

Achievement  Test  Data 

A summary  of  posttest  means  and  adjusted  means  is  provided  in  Table  1 . Student  pretest  results  showed  no 
significant  differences  between  the  experimental  group  and  the  control  group. 

Table  1 . 

Mean  Adjusted  Posttest  Scores  for  Achievement  Tests  bv  Group  and  Achievement  Level. 


Group 


Achievement  Level 
Low High Total 


Experimental 


M 

56.10 

51.61 

53.75 

Unadj.  Mean 

43.71 

62.17 

53.36 

Control 

M 

48.50 

49.67 

49.14 

Unadj.  Mean 

40.18 

59.80 

49.52 
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The  adjusted  mean  for  students  in  the  experimental  group  (M  = 53.75)  was  significantly  higher  than  the  adjusted 
mean  for  students  in  the  control  group  (M  = 49.14),  F(l,64)  = 4.76,  p < .05  (see  Figure  1).  The  group  X ability  level 
interaction  effect  showed  the  greatest  disparity  of  scores  between  the  low-ability  students  in  the  experimental  group  and 
low-ability  students  in  the  control  group.  The  adjusted  mean  for  students  in  the  experimental/low-ability  group  was 
56.10,  whereas  the  adjusted  mean  for  students  in  the  control/low-ability  group  was  48.50  (see  Figure  2).  However,  this 
difference  was  not  significant,  F(l,64)  = 1.67,  p = .20. 

Figure  1. 

Adjusted  Posttest  Scores  bv  Group. 

Adjusted  Posttest  Scores  by  Group 


Experimental 


Control 


Group 


Figure  2. 

Adjusted  Posttest  Scores  by  Ability  Level. 


Adjusted  Post  test  Scores  by  Ability  Level 


Low  Ability  High  Ability 
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Questionnaire  Results 

Responses  to  the  questionnaire  revealed  attitudinal  differences  between  students  in  the  control  group  and  students 
in  the  experimental  group,  particularly  in  terms  of  attitudes  towards  math  and  the  computer  math  lessons.  Refer  to  Table 
2 for  a summary  of  the  questionnaire  results. 

Table  2. 

Student  Exit  Questionnaire  Results, 


Questionnaire  Statement  Student  Responses 


Experimental 

Control 

1.  Do  you  like  math? 

Yes  = 91% 

Yes  = 57% 

2.  Do  you  think  you  do  well  in  math? 

Yes  = 86% 

Yes  = 76% 

3.  Do  you  like  the  computer  math 
lessons? 

Yes  = 94% 

Yes  = 43% 

4.  Do  the  computer  math  lessons  help 
you  with  your  math  classwork? 

Yes  = 85% 

Yes  = 62% 

5.  Do  the  computer  math  lessons  help 
you  with  your  math  homework? 

Yes  = 73% 

Yes  = 48% 

6.  Would  you  rather  work  on  the  math 
lessons  alone  or  with  a partner? 

Parti  ?r  = 68% 

Partner  = 76% 

On  the  statement  dealing  with  math  attitudes  (question  1),  91%  of  the  students  in  the  experimental  group 
responded  that  they  liked  math,  as  opposed  to  57%  in  the  control  group.  A chi  square  test  of  these  data  revealed 
significant  differences,  %2  (1,N  = 65)  = 10.13,  p<  .002. 

In  terms  of  the  computer  math  lessons  (question  3),  94%  of  the  students  in  the  experimental  group  responded 
that  they  liked  the  computer  math  lessons,  as  opposed  to  only  43%  in  the  control  group.  Once  again,  a chi  square  test 
revealed  significant  differences,  %2  (1,  N = 65)  = 20.28,  p < .001 . 

In  addition,  students  in  the  experimental  group  responded  significantly  more  favorably  than  students  in  the 
control  group  when  asked  if  they  believed  the  computer  math  lessons  helped  them  with  their  math  classwork  and 
homework.  Eighty-five  percent  of  the  students  in  the  experimental  group  responded  that  they  believed  the  computer  math 
lessons  helped  them  with  their  math  classwork  (question  4),  as  opposed  to  62%  of  the  students  in  the  control  group,  %2 
( 1 , M = 65)  = 3.94,  p < .05.  Seventy-three  percent  of  the  students  in  the  experimental  group  responded  that  they  believed 
the  computer  math  lessons  helped  them  with  their  math  homework  (question  5),  compared  to  48%  of  the  students  in  the 
control  group,  y2  (1, 14=  65)  = 3.92,  p < .05. 

On  the  questionnaire  statement  dealing  with  working  with  a partner  (question  6),  76%  of  the  students  in  the 
control  group  stated  that  they  would  rather  work  on  the  computer  with  a partner,  as  opposed  to  68%  in  the  experimental 
group.  These  data  were  not  statistically  significant,  however. 

The  attitudinal  data  were  also  analyzed  using  student  ability  level  as  a blocking  variable.  This  analysis  revealed 
no  additional  significant  differences  between  the  groups. 
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Computer  Lab  Observations 

During  each  computer  lab  observation,  the  researcher  recorded  verbal  comments  made  by  students  to  other 
students,  the  teacher  or  lab  assistant,  or  themselves.  Comments  recorded  for  students  in  the  experimental  group  dealt 
mainly  with  group  interactions.  Examples  of  some  of  the  comments  recorded  included  phrases  such  as:  "Do  you  agree? 

",  "If  you're  only  watching  me  you're  not  going  to  learn!",  "Pay  attention!",  "You  do  this  one.",  "Don't  ask  for  help 
yet",  and  "We've  got  to  take  turns!".  In  addition,  the  researcher  noted  that  many  of  the  questions  asked  by  students  in  the 
experimental  group  dealt  with  the  content  in  the  computer  math  lessons  (e.g.,  "We  don't  understand  this  problem.",  "Do 
we  need  to  add  or  multiply  here?"). 

The  comments  made  by  students  in  the  control  group  dealt  mainly  with  students'  frustration  with  the  computer 
math  lessons.  A sample  of  some  of  their  statements  included:  "Can  I get  a sledgehammer?",  "I'm  getting  tired  of  this 
junk!",  "I  want  to  go  home.",  "I  don't  know  and  I don't  care!",  "I  hate  math.",  "What  time  do  we  leave?",  and  "I'm 
stupid!". 

In  addition  to  recording  students’  comments,  the  researcher  also  collected  anecdotal  data  regarding  interactions 
between  students  working  in  cooperative  groups,  on-task  and  off-task  behaviors  exhibited  by  students,  and  interactions' 
between  students  and  the  teacher  or  lab  assistant.  The  researcher  observed  that  students  in  the  experimental  group 
demonstrated  behaviors  designed  to  monitor  each  other  and  keep  each  other  on  task.  For  example,  if  one  of  the  students 
in  a cooperative  group  was  not  paying  attention,  the  other  student  in  the  group  would  gain  the  student's  attention,  either 
through  a verbal  comment  or  a tactile  cue  (such  as  pulling  on  their  shirt),  and  would  re-engage  them  in  the  lesson.  In 
this  way,  the  members  of  the  groups  were  observed  assisting  each  other  in  remaining  on-task  and  engaged  in  the 
instructional  content. 

Analysis  of  observational  data  of  students  in  the  control  group  revealed  a different  set  of  behaviors  than  those  in 
the  experimental  group.  Students  in  the  control  group  repeatedly  demonstrated  off-task  behaviors  such  as  putting  their 
heads  down  on  the  table  and  giving  up,  quitting  out  of  the  lessons  before  they  were  finished  to  work  on  writing  or 
drawing  activities;  staring  out  the  window,  asking  to  go  to  the  bathroom  or  go  get  a drink,  and  "guessing"  on  math 
activities  just  to  get  through  the  lessons.  These  types  of  behaviors  were  generally  not  exhibited  by  students  in  the 
experimental  group.  The  lab  assistant  was  repeatedly  overheard  explaining  to  students  that  they  needed  to  ask  for  help 
when  they  were  stuck  on  a problem,  and  not  give  up.  It  was  apparent  that  some  students  in  the  control  group  had 
difficulty  knowing  when  to  ask  for  help. 


DISCUSSION 

The  notion  of  combining  ILS-delivered  instruction  with  cooperative  learning  activities  has  not  been  widely 
studied.  This  research  has  shown  that  integrating  cooperative  learning  with  ILS-delivered  instruction  is  an  effective 
instructional  strategy.  Results  of  this  study  showed  that  students  working  on  math  activities  delivered  by  an  ILS 
performed  better  on  tests  of  those  activities  when  they  completed  the  activities  in  cooperative  pairs  than  when  they 
worked  on  the  activities  individually.  In  addition,  this  study  showed  that  combining  a cooperative  learning  model  with 
ILS  instruction  had  a positive  impact  on  student  attitudes  towards  the  instructional  content. 

There  are  a number  of  possible  reasons  for  the  significant  achievement  differences  between  the  experimental 
group  and  the  control  group.  One  explanation  is  that  students  working  in  pairs  were  able  to  assist  one  another  with  the 
computer  math  activities,  thus  creating  peer  support  structures  within  their  groups  (Mevarech,  1994).  If  one  student  in 
the  pair  did  not  understand  the  math  concept  being  presented,  the  other  student  had  the  opportunity  to  explain  the  concept. 
Through  this  dialog,  not  only  did  students  who  did  not  understand  the  concepts  receive  additional  instruction  from  their 
partners,  but  their  partners  may  have  been  able  to  reinforce  their  own  understanding  of  the  concepts  through  explanation. 
If  students  working  individually  did  not  understand  any  of  the  concepts  being  presented,  the  alternatives  were  to  either 
guess  at  the  answer  ■ >r  ask  the  teacher  or  lab  attendant  for  assistance.  If  they  chose  to  ask  for  assistance,  they  generally 
had  to  wait  for  the  U acher  to  finish  helping  another  student,  thus  decreasing  their  on-task  time  further.  If  they  chose  to 
guess,  then  it  was  likely  that  they  did  not  understand  the  concept  and  would  have  more  difficulty  understanding  future 
concepts. 

A second  explanation  is  that  students  working  in  pairs  tended  to  stay  on  task  while  working  on  the  lessons, 
whereas  students  working  individually  were  more  likely  to  either  be  distracted  from  the  math  activities  or  stop  working 
altogether.  During  computer  lab  observations,  the  researcher  recorded  that  students  in  the  conh  1 group  demonstrated  off- 
task  behaviors  such  as  putting  their  heads  down  on  the  desk  or  staring  out  the  window.  These  behaviors  were  not 
observed  in  the  experimental  classes.  By  staying  focused  on  the  computer  math  lessons  and  ^igaged  in  the  content  of  the 
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lessons,  students  in  the  experimental  group  most  likely  learned  more  math  concepts,  and  thus  were  able  to  perform  better 
on  the  math  achievement  test. 

Finally,  it  is  likely  that  teachers  were  able  to  spend  more  time  with  and  provide  a higher  quality  of  assistance  to 
students  working  in  cooperative  groups  than  students  working  individually.  Even  though  there  were  about  the  same 
number  of  students  in  each  of  the  classes  participating  in  the  study,  teachers  for  the  experimental  group  classes  had  an 
easier  time  getting  around  to  all  the  students  that  needed  assistance.  This  is  because  (1)  in  the  experimental  classes, 
teachers  could  work  with  two  students  at  the  same  time,  whereas  in  the  control  cisss  the  teacher  had  to  work  with  each 
student  individually,  and  (2)  there  were  fewer  off-task  behavior  problems  that  required  teacher  intervention  in  the 
experimental  classes.  Thus,  pairing  of  students  allowed  the  teacher  to  spend  more,  time  with  the  students  who  needed 
help  and  thus  provide  a greater  amount  of  small-group  tutoring  on  difficult  concepts.  This  may  have  been  particularly 
effective  for  low-achieving  students,  who  had  a greater  need  for  teacher  intervention  and  guidance  than  high-achieving 
students. 

Results  of  this  study  also  demonstrated  significant  differences  in  attitudes  towards  both  the  content  and  the 
delivery  system.  As  with  the  achievement  results,  a major  contributor  to  the  positive  attitudes  reported  by  students 
working  in  cooperative  groups  could  be  the  support  structure  provided  by  the  cooperative  groups.  Students  working 
individually  on  the  lessons  were  observed  getting  frustrated  and  upset  with  the  computer  math  activities,  whereas  students 
working  on  the  lessons  in  cooperative  groups  were  observed  helping  one  another  stay  focused  on  the  activities.  Without 
the  availability  of  peer  assistance,  students  in  the  control  group  had  to  struggle  through  the  lessons  alone.  This  isolation 
from  their  peers  may  have  contributed  to  their  dislike  of  the  lessons  and  of  math  in  general.  Also,  students  in  the  control 
group  possibly  did  not  receive  as  much  teacher  support  in  the  computer  lab  when  they  were  having  difficulty  with  the 
math  activities,  thus  exacerbating  their  frustration  and  contributing  to  their  lack  of  success  in  the  lab.  It  is  logical  to 
assume  that  students  would  have  negative  attitudes  towards  activities  in  which  they  were  not  succeeding. 

Limitations 

Readers  should  be  careful  when  interpreting  the  results  of  this  study,  and  to  take  into  account  the  following 
limitations  which  may  have  impact  on  the  internal  and  external  validity  of  these  results: 

1.  Subjects  and  sample  size  - only  fifth-grade  students  from  one  elementary  school  participated  in  this  study.  It 
may  be  difficult  to  generalize  these  results  to  other  age-groups  For  example,  younger  students  may  have  had  more 
difficulty  interacting  and  staying  on-task  in  cooperative  group  activities.  In  addition,  only  65  c*jdents  participated  in  this 
study.  The  researcher  concedes  that  the  results  may  have  been  more  significant  with  a larger  population  participating,  due 
to  larger  sub-groups  and  the  increase  in  statistical  power  associated  with  a larger  population. 

2.  Subject-matter  - mathematics  was  used  as  the  content  area  for  this  study.  These  results  may  not  necessarily 
transfer  to  other  subject  areas  such  as  language  arts,  science,  or  social  studies.  Writing  activities,  for  example,  generally 
take  longer  to  complete  than  math  problems  and  students  may  have  more  difficulty  completing  this  type  of  activity  in 
cooperative  groups  due  to  the  personal  nature  of  many  writing  activities. 

3.  Teaching  styles  - readers  should  be  reminded  that  the  control  group  consisted  of  all  the  students  in  one 
particular  fifth-grade  class,  and  that  the  experimental  group  consisted  of  students  in  two  fifth-grade  classes.  The  researcher 
concedes  that  this  method  of  dividing  the  experimental  group  and  the  control  group  is  not  as  experimentally  sound  as  a 
true  random  dispersion  of  students  due  to  confounding  variables  (such  as  the  teacher)  that  could  not  be  controlled. 
However,  due  to  ethical  considerations,  the  researcher  felt  that  the  method  chosen  was  the  best  way  of  dividing  the 
students.  Students  in  each  of  the  participating  classrooms  did  not  have  the  same  teachers  for  math  instruction.  Thus,  it 
was  inevitable  that  different  teaching  styles,  discipline  strategies,  and  instructional  methods  would  be  used  in  the  different 
classrooms.  These  differences  in  teaching  styles  during  computer  lab  time  most  likely  had  some  impact  on  the  results  of 
this  study.  Readers  should  take  this  into  account  when  generalizing  these  results. 

Conclusion 

Research  investigating  both  integrated  learning  systems  and  cooperative  learning  has  shown  them  to  have 
positive  effects  on  academic  achievement  levels  and  attitudes  of  students.  This  study  has  revealed  that  combining  an  ILS 
delivery  system  with  cooperative  learning  strategies  can  also  facilitate  the  learning  process. 

There  are  several  considerations  that  should  be  taken  into  account  when  combining  cooperative  learning  and 
integrated  learning  systems.  First,  a learning  area  conducive  to  cooperative  learning  activities  is  important,  particularly 
in  a computer  lab  setting.  Students  need  to  be  able  to  easily  work  together  in  their  group  without  feeling  cramped,  and 
groups  need  to  be  separated  from  one  another  so  that  they  do  not  distract  each  other.  Second,  teacher  support  in  the 
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computer  lab  is  key  to  the  successful  combination  of  cooperative  learning  and  ILS.  Cooperative  learning  provides 
teachers  with  the  opportunity  to  spend  more  time  tutoring  small  groups  of  students.  However,  teachers  need  to  take  an 
active  instructional  role  in  the  computer  lab  in  order  for  this  strategy  to  have  the  greatest  impact  on  students.  Teachers 
who  take  a passive,  reactive  role  in  the  computer  lab  will  most  likely  see  that  cooperative  learning  and  ILS  provides 
fewer  positive  academic  benefits  to  their  students.  Finally,  the  activities  students  are  completing  using  the  ILS,  whether 
those  activities  are  completed  in  cooperative  groups  or  individually,  need  to  correspond  to  the  concepts  being  taught  in 
the  classroom.  If  students  learn  new  concepts  in  the  computer  lab,  and  those  concepts  are  not  reinforced  in  the 
classroom,  then  students  will  have  difficulty  remembering  those  concepts.  Teachers  need  to  use  the  instructional 
activities  provided  through  the  ILS  to  enhance  and  support  their  curriculum.  Keeping  classroom  activities  and  computer 
lab  activities  separate  and  distinct  will  diminish  the  effectiveness  of  the  instruction  provided  in  both  settings. 

The  results  of  this  study  suggest  to  educators  that  using  cooperative  learning  strategies  in  a computer  lab  setting 
can  have  greater  benefits  than  increased  standardized  achievement  test  scores.  Although  test  results  did  reveal  significant 
differences  between  the  experimental  group  and  the  control  group,  the  differences  between  the  groups  were  greater  than 
those  measur  id  by  a test.  Student  behaviors  appeared  to  be  markedly  different,  and  the  enjoyment  and  excitement 
observed  in  the  experimental  classes  was  not  as  noticeable  in  the  control  class.  These  behavior  differences  should 
encourage  educators  to  look  beyond  simple  test  scores  in  order  to  determine  the  full  merits  of  this  strategy. 

This  study  provides  evidence  that  an  instructional  strategy  that  research  has  demonstrated  as  effective  with 
different  populations  of  students  can  be  effectively  integrated  into  a ILS  lab  setting.  Based  on  these  results,  educators 
may  want  to  consider  experimenting  with  the  use  of  other  instructional  strategies  in  ILS  labs.  Combining  different 
strategies  with  ILS  instruction  will  definitely  be  more  effective  then  continuing  to  use  ILS  labs  as  reinforcement  tools  or 
as  instruments  to  passively  feed  instruction  to  students.  An  integrated  learning  system  is  a powerful  tool  in  its  own  right 
when  used  properly.  Utilizing  effective  instructional  strategies  such  as  cooperative  learning  with  an  ILS  can  make  that 
tool  even  more  powerful. 
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The  Graphical  User  Interface  (GUI)  has  become  something  of  a standard  for  instructional  programs  in  recent  years.  Not  all 
graphical  programs’  interfaces,  however,  implement  the  GUI  in  the  same  way.  Some  are  more  graphical  than  others  and 
some  employ  a more  unified  approach  than  others.  One  type  of  graphical  user  interface  is  the  metaphorical  GUI.  A 
metaphorical  GUI  bases  its  look  and  feel,  as  well  as  many  of  its  operations,  on  an  explicit  or  implicit  metaphor.  For 
example,  the  Macintosh  GUI  is  based  on  the  “desktop”  metaphor;  objects  one  manipulates  within  the  GUI  are  implied  to 
be  objects  one  might  well  find  on  a real  office’s  desktop. 

Cates  (1994)  divided  metaphors  into  two  classes:  underlying  (or  primary)  metaphors  and  auxiliary  (or  secondary) 
metaphors.  The  underlying  metaphor  is  the  principal  metaphor  employed  or,  by  default,  the  first  metaphor  introduced, 
while  an  auxiliary  metaphor  is  a subsequent  metaphor  employed  by  the  product.  The  underlying  metaphor  establishes  the 
implied  comparison  and  the  basic  framework  for  the  interface.  Auxiliary  metaphors  may  be  either  complementary  or 
confounding.  Complementary  auxiliary  metaphors  add  to,  enhance,  and  help  complete  the  underlying  metaphor.  They 
exhibit  high  degrees  of  affinity  and  acceptable  levels  of  contrast  in  relation  to  the  underlying  metaphor  (Kittay,  1987)  and 
they  operate  in  ways  that  match  the  experiences  of  users  working  with  the  object  to  which  the  designer  is  comparing  the 
product's  interface  (the  underlying  metaphor).  When  contrast  occurs,  it  is  of  the  type  that  encourages  users  to  reinterpret 
and  reconstruct  their  understanding  of  how  the  interface  operates  (Miller,  1979;  Searle,  1979).  A confounding  auxiliary 
metaphor  is  one  in  which  the  user  finds  the  contrast  too  great  and  rejects  the  comparison  (Semper,  1990). 

Once  again  using  the  Mac’s  operating  system  as  the  basis  for  discussion,  the  underlying  metaphor  is  the  “desktop.”  The 
main  auxiliary  metaphors  are  the  document,  the  folder,  and  the  trash  can.  Documents  — represented  by  icons  that  look 
like  papers  — and  folders  are  clearly  objects  one  might  find  on  a desktop  and  are  complementary  auxiliary  metaphors, 
therefore.  In  contrast,  however,  the  trash  can  auxiliary  metaphor  would  need  to  be  classified  as  confounding.  While  one 
might  well  expect  to  find  a trash  can  in  an  office,  it  is  both  illogical  and  inconsistent  — although  convenient  — for  the 
trash  can  to  reside  on  top  of  the  desk  (Vertelney,  Arent,  & Lieberman,  1990). 


Not  all  objects  or  images  that  appear  in  a graphical  user  interface  are  guaranteed  to  be  metaphorical,  however.  For 
example,  once  again  in  the  Mac  interface,  while  a computer  hard  disk  (represented  in  the  interface  by  a labeled  icon  of  a 
hard  drive)  is  clearly  an  object  that  might  appear  on  a desktop,  it  is  in  no  way  metaphorical.  The  same  is  true  for  floppy 
disks  (represented  similarly  by  labeled  iconic  representations  on  the  desktop).  That  is,  in  order  for  something  to  be 
metaphorical,  it  must  represent  one  thing  in  terms  of  something  else  different  (Black,  1979;  Lakoff  & Johnson,  1980).  In 
addition,  interfaces  sometimes  employ  graphical  images  that  are  purely  symbolic.  For  example,  the  Mac  interface 
displays  what  appears  to  be  a teddy  bear’s  head  attached  to  a 3.5"  diskette  “body”  to  represent  the  Fetch™  program.  While 
the  interface  designers  at  Apple  cannot  be  blamed  for  non-metaphorical  program  icons,  there  is  reason  to  be  concerned 
that  non-metaphorical  icons  may  serve  to  “break  the  spell”  of  the  underlying  metaphor,  thus  reducing  the  user  to  a non- 
systematic  evaluation  of  what  he  or  she  sees  on  a case-by-case  basis  (Cates,  1993).  When  the  spell  is  broken,  the  user  no 
longer  seems  likely  to  benefit  from  the  augmentative  properties  of  metaphorical  interface  design  that  Rosendahl-Kreitman 
(1990)  has  suggested  add  so  much  to  the  experience  of  using  the  program.  In  a graphical  user  interface,  metaphorical 
images  can  help  users  make  linkages  between  their  imaginal  and  semantic  networks  (Paivio,  1971,  1979,  1986),  thereby 
creating  the  webs  of  association  that  help  to  take  advantage  of  learners’  mental  models  and  previous  experiences 
(Erickson,  1990;  Mac  Cormac,  1985). 

Cates  (1993)  offered  a three-level  (superficial-literal-interpretive)  model  for  evaluating  the  metaphorical  promise  of  icons. 
That  paper  emphasized  the  relationship  between  visual  images  and  metaphors  in  terms  of  the  ways  users  understand  and 
interpret  interfaces.  Cates  (1994)  suggested  that  designers  use  the  five-part  POPIT  (properties,  pperations,  phrases, 
images,  and  lypes)  model  to  identify  imaginal  and  semantic  links  and  thus,  complementary  auxiliary  metaphors.  The 
present  paper  attempts  to  classify  metaphorical  interfaces  along  a four-point  continuum  from  lowest  level  of  metaphorical 
implementation  to  highest,  from  non-metaphorical  to  immersive.  It  explores  how  position  on  that  continuum 
exemplifies  the  interaction  between  complementary  and  confounding  auxiliary  metaphors  and  determines  the  design 
demands  of  an  interface.  Along  the  way,  we  will  consider  how  the  same  basic  metaphor  may  be  implemented  differently 
to  meet  these  demands. 
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Graphical  But  Non-Metaphorical  User  Interfaces  (GNUI) 

This  position  on  the  continuum  equates  with  the  null  or  end-stop  position.  That  is,  it  is  included  here  merely  to  indicate 
where  metaphorical  GUIs  end.  A GNUI  is  an  interface  that  employs  images  and  asks  the  user  to  perform  actions  using 
graphical  images,  but  employs  no  underlying  metaphor.  That  is,  the  user  is  not  asked  to  compare  what  he  or  she  is  doing 
to  actions  in  some  other  situation,  setting,  or  condition. 


Figure  6:  A sample  Graphical  Non-Metaphorical  User  Interface  (GNUI). 


For  instance,  figure  1 shows  a possible  GNUI  implementation  of  a history  lesson  on  biography  and  being  remembered. 
As  the  figure  shows,  five  figures  to  be  examined  are  represented  by  abstract  (cartoon)  drawings.  Navigation  functions  are 
represented  by  pointing  fingers  and  an  exit  sign.  While  these  have  some  minor  metaphoric  properties,  in  the  absence  of 
an  underlying  metaphor,  these  properties  will  most  probably  go  unnoticed  by  users.  TL lie  television  screen,  audio  cassette 
tape,  filing  cabinet,  and  diskette  are  clearly  visual  representations  or’  the  objects  themselves,  not  metaphors.  The  same  is 
true  of  the  pencil  and  printer.  One  might  try  to  read  something  metaphoric  into  the  glossary  function  and  the  help 
function  here,  but  once  again,  in  the  absence  of  an  underlying  metaphor,  these  icons  are  primarily  graphical  or 
representational. 

Mixed  Metaphorical  Graphical  User  Interface  (Mixed  MGUI) 

A Mixed  MGUI  would  employ  an  underlying  metaphor,  but  would  then  employ  a variety  of  auxiliary  metaphors  that 
were  confounding.  Figure  2 shows  such  a Mixed  MGUI. 
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Click  on  any  gravestone  below 
to  learn  more  about  that  person: 


.■When  >oii  click  on 
a <*ra.\  eslone  un  the 
left,  'informal  ion 
about  that  pet  s«'ih 
hi  If  appear  here. 


Figure  2:  A sample  Mixed  Metaphorical  Graphical  User  Interface. 

Notice  in  the  figure  that  the  underlying  metaphor  is  of  a cemetery,  although  the  metaphor  is  more  implied  than  explicit, 
since  we  do  not  see  the  entire  cemetery  at  any  point.  The  gravestones  are  certainly  consonant  with  the  cemetery 
metaphor.  Trucks  for  navigation  and  a lighted  exit  sign  are  not,  however.  It  is  unclear  how  a notebook  or  a toolbox 
complements  the  metaphor,  although  one  might  envision  ways  to  make  them  consonant.  This  is  part  of  the 
rcinterpretative  process  that  users  perform  in  order  to  make  metaphorical  images  fit  (Miller,  1979).  The  designer  has 
asked  them,  however,  to  make  a bit  of  a stretch.  One  might  think  of  this  as  stretching  credibility.  That  is,  as  the  user 
sees  more  images  that  exhibit  high  levels  of  dissonance,  he  or  she  begins  to  wonder  whether  the  metaphor  is  appropriate. 
Once  users  reject  the  metaphor,  they  begin  to  think  of  images  on  the  screen  as  devices  only,  not  elements  in  a larger 
system  that  permit  the  user  to  infer  and  construct  new  understandings  based  on  logical  extensions  of  the  metaphor  itself. 
While  images  and  icons  of  toolboxes,  notebooks,  and  exit  signs  might  represent  reasonable  auxiliary  metaphors  for  the 
operations  they  provide  or  empower,  in  this  case  they  clearly  do  not  complement  the  underlying  metaphor. 


Another  variation  on  the  Mixed  MGUI  is  an  interface  in  which  there  is  an  underlying  metaphor  and  some  complementary 
auxiliary  metaphors,  but  some  images  are  non-metaphorical  or  represent  confounding  auxiliary  metaphors.  For  instance, 
by  added  images  of  a printer,  a computer  disk,  and  other  similar  devices  to  the  images  in  Figure  2,  we  could  confound  the 
metaphor  further.  Even  if,  at  the  same  time,  we  added  additional  complementary  auxiliary  metaphors,  it  seems  likely  that 
users  would  be  inclined  to  reject  the  underlying  metaphor,  defaulting  instead  to  seeing  everything  on  the  screen  as  a 
device.  In  fact,  the  presence  of  non-metaphorical  images  seems  likely  to  encourage  them  to  view  screen  images  as  simply 
devices. 


Still  another  variation  on  the  Mixed  MGUI  is  one  in  which  the  designer  has  employed  more  than  one  underlying 
metaphor,  accompanied  by  — or  resulting  in — confounding  auxiliary  metaphors.  Thus,  when  the  user  goes  to  certain 
portions  of  the  program,  the  underlying  metaphor  changes.  For  instance,  imagine  that  the  initial  underlying  metaphor 
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were  of  a cemetery  accompanied  by  complementary  auxiliary  metaphors.  When  the  user  clicks  on  some  image,  he  or  she 
is  sent  to  a music  store  to  listen  to  music  that  is  in  some  way  related.  In  effect,  the  underlying  metaphor  has  changed;  a 
music  store  is  not  a cemetery.  Consider  what  would  happen  if  the  designer  retained  some  or  all  of  the  same  icons 
provided  when  the  underlying  metaphor  was  of  a cemetery.  Since  the  complementary  metaphors  for  a cemetery  are  likely 
to  be  confounding  metaphors  in  a music  store,  we  would  say  that  the  designer  had  mixed  metaphors. 

There  are  too  few  clearly  explicated  examples  in  the  field  of  interface  design  and  many  of  the  examples  available  are  not 
as  useful  or  well  designed  as  they  might  be  (Mountford,  1990).  It  is  often  easier  to  focus  on  simple  graphics  and  pleasing 
layout  than  on  making  the  metaphor  work.  In  fact.  Nelson  (1990)  suggested  that  an  overdependence  on  metaphor  can  ruin 
an  otherwise  good  design.  At  the  same  time,  making  the  most  of  a metaphor  may  enable  designers  to  imply  more  than 
they  state  and  may  help  users  comprehend  without  having  to  tell  them  all  details  (Andre  & Phye,  1986).  The  absence  of 
adequate  explication  and  the  paucity  of  strong  examples  may  explain  why  so  many  designers*  GUIs  suffer  from  the  mixed 
metaphor  problem. 

One  way  to  prevent  auxiliary  metaphors  from  mixing  dissonantly  with  the  underlying  metaphor  might  be  to  employ  the 
POPIT  model  (Cates,  1994)  to  identify  complementary  auxiliary  metaphors.  To  eliminate  the  problem  of  mixing 
underlying  metaphors,  designers  could  look  for  a larger  unifying  primary  metaphor.  For  instance,  a clever  designer  might 
be  able  to  work  around  the  cemetery/music  store  problem  in  a couple  of  ways.  ^ ° first  would  be  to  make  the  cemetery 
not  the  underlying  metaphor,  but  rather  itself  an  auxiliary  metaphor  for  a larger  geographic  primary  metaphor.  A second 
way  would  be  to  couple  some  sort  of  transitional  device,  either  visual  or  navigational  that  conveyed  the  sense  of  going 
from  one  location  to  another  with  a changed  set  of  auxiliary  metaphors  that  complement  the  demands  of  the  new 
location.  Keys  here  would  appear  to  be  making  the  transition  easily  comprehensible  and  making  the  set  of  auxiliary 
metaphors  in  the  new  location  comparable  in  both  function  and  location  to  the  set  provided  in  the  previous  location. 

Thematic  Metaphorical  Graphical  User  Interface  (Thematic  MGUI) 

A Thematic  MGUI  is  an  interface  in  which  the  auxiliary  metaphors  are  all  complementary  to  the  underlying  metaphor. 
The  icons  and  images  employed  all  support  the  underlying  metaphor  and  help  to  enhance  its  believability  and  efficacy. 
That  is,  there  is  a consistent  theme  exemplified  by  the  interface  and  all  aspects  of  the  interface  appear  to  integrate  this 
theme.  Figures  3 and  4 illustrate  how  a Thematic  MGUI  might  be  used  in  our  exemplary  history  lesson. 


Click  on  the  gate  to 
enter  the  cemetery: 


Figure  3:  The  entrance  screen  for  a sample  Thematic  MGUI. 
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The  underlying  theme  of  a cemetery  continues  to  serve  as  the  primary  metaphor.  The  program’s  entrance  (figure  3)  and 
departure  are  accomplished  through  the  use  of  a gate;  once  again,  more  similar  to  the  way  in  which  one  actually  gains 
access  to  a cemetery.  The  sample  Thematic  MGUI  in  figure  4 uses  icons  that  complement  the  underlying  metaphor.  They 
are  objects  or  functions  one  might  expect  to  find  in  a cemetery.  Navigation  is  now  based  on  hearses.  Clicking  on 
tombstones  elicits  biographical  data.  Some  gravesites  have  larger  memorials,  statues,  or  mausoleums.  Clicking  on  large 
memorials  and  statues  supplies  information  about  why  the  deceased  is  better  known  or  more  impressively 
commemorated.  Clicking  on  mausoleums  provides  information  about  famous  families  of  deceased  persons.  Some 
gravesites  carry  markers  indicating  military  service  and  some  have  flowers.  Clicking  on  military  markers  makes  it 
possible  for  the  user  to  gain  information  about  the  person’s  military  record.  Clicking  on  gravesites  with  flowers  allows 
the  user  to  find  out  why  this  person  is  remembered.  Some  sites  may  have  a person  visiting  the  grave.  Clicking  on  this 
person  enables  the  user  to  gain  more  personal  information  about  the  deceased. 
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Figure  4:  A sample  Thematic  Metaphorical  Graphical  User  Interface. 

Auxiliary  metaphors  go  beyond  grave-side  information,  however.  Clicking  on  a small  building  on  the  right  side  of  the 
screen  provides  access  to  a caretaker  and  cemetery  records.  The  caretaker  might  serve  as  a guide  or  advisor,  helping  the 
user  decide  what  to  do  (Oren,  Salomon,  Kreitman,  & Don,  1990).  Clicking  on  the  record  book  for  the  cemetery  could 
bring  up  a plot  map  of  who  is  buried  where  and  a brief  identification  of  that  person,  in  a fashion  similar  to  what  might 
be  done  for  a real  cemetery.  There  could  be  funeral  parties  on  which  one  might  click  also.  Some  gravesites  are  clearly 
older  than  others  with  faded  or  weather-worn  gravestones,  while  more  recent  gravesites  have  newer  stones  or  may  have 
disturbed  soil.  In  each  case,  we  are  using  aspects  of  the  underlying  cemetery  metaphor  to  suggest  appropriate 
complementary  metaphors,  while  at  the  same  time  employing  conventions  to  convey  recency  to  users  (Brown,  1988). 

Thematic  metaphorical  GUIs  are  not  without  their  problems.  For  instance,  navigation  can  present  a challenge.  In  our 
example,  we  have  maintained  lateral  navigation  across  screens  in  a largely  linear  fashion.  If  we  were  to  use  hyper- 
navigation  — that  is,  navigation  by  linking  from  one  image  to  another  across  multiple  screens — users  could  quickly 
become  disoriented.  This  could  prove  particularly  true  if  the  links  took  users  to  screens  in  which  the  underlying  metaphor 
were  different.  In  our  example,  one  could  use  the  plot  map  in  the  caretaker’s  house  as  a navigational  device  to  permit  the 
user  to  move  about  the  cemetery.  This  would  make  it  possible  to  eliminate  the  back  and  go  on  icons,  unless  users  were 
required  to  navigate  across  metaphorical  screens. 
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Maintaining  context  and  a sense  of  control  are  also  issues  here.  Users  need  to  be  able  to  take  advantage  of  crucial  features 
regardless  of  where  navigational  actions  have  taken  then.  Thus,  in  our  example,  moving  from  gravesite  to  gravesite  using 
the  cemetery  map  should  not  leave  the  user  without  access  to  navigation  and  other  key  functions.  One  approach  could  be 
to  maintain  certain  things  in  common  locations,  regardless  of  the  setting.  For  example,  in  the  cemetery  metaphor,  the 
caretaker’s  building  could  always  be  visible,  regardless  of  the  gravesite  visited.  For  instance,  the  cemetery  might  be  laid 
out  in  such  a fashion  that  the  caretaker’s  building  were  a central  device  with  gravesites  radiating  out  from  it  like  spokes 
from  a wheel  (see  figures  5 - 8). 


Figures  5-8:  Four  views  of  the  caretaker  house. 


Immersive  Metaphorical  Graphical  User  Interfaces  (Immersive  MGUI) 

While  thematic  GUIs  should  help  to  enhance  the  believability  and  predictability  of  the  program,  they  still  tend  to  be 
“flat.”  That  is,  they  tend  to  be  based  on  back-and-forth  or  simple  bidirectional  navigation  among  sites  or  locations.  The 
next  step  in  GUI  design  would  be  to  add  the  third  dimension.  In  an  Immersive  MGUI,  users  would  find  themselves 
working  through  an  interface  that  consists  largely  of  three-dimensional  representations,  creating  an  interactive 
environment  for  the  user.  Some  have  termed  such  an  environment  a “microworld”  (Rieber,  1992).  Many  of  the  objects  in 
this  environment  might  be  manipulatable.  For  example,  one  might  be  able  to  pick  objects  up  and  examine  them  from 
different  viewpoints.  One  might  be  able  to  move  things  from  one  location  to  another  in  a fairly  realistic  fashion. 

Although  the  Thematic  MGUI  in  figure  4 exhibits  some  three-dimensional  properties,  it  does  not  provide  much  in  the 
way  of  manipulation.  Users  click  on  labeled  tombstones,  mausoleums,  statues,  and  the  caretaker  house.  What  they  do  not 
do  is  manipulate  and  navigate  dynamically  in  the  third  dimension.  In  an  Immersive  MGUI,  users  might  manipulate  and 
reconstruct  broken  tombstones  from  pieces  found  lying  on  the  ground.  They  might  be  asked  to  find  missing  gravesites  or 
missing  bodies  by  using  information  that  was  available  through  manipulation,  perhaps  even  through  exhumation 
(accompanied  by  appropriate  ethical,  legal,  and  religious  actions).  The  devices  offered  by  the  Immersive  MGUI  might 
include  such  tools  as  shovels,  metal  detectors,  sonic  finders,  even  bulldozers  and  backhoes.  Instead  of  simply  clicking  on 
objects,  usually  in  isolation,  users  do  more  dragging  and  dropping  and  more  combinatorial  activities.  Thus,  for  instance, 
instead  of  clicking  on  a tombstone  or  a box  of  charcoals  to  get  a rubbing,  users  must  drag  an  appropriate  size  piece  of 
tracing  paper  to  the  stone,  attach  it  properly,  obtain  a charcoal,  and  then  rub  the  charcoal  from  side  to  side  across  the 
stone  until  a satisfactory  image  is  obtained. 

Two  key  components  of  Immersive  MGUIs  are  the  use  of  three-dimensional  objects  and  images  and  the  provision  and 
expectation  of  enhanced  manipulatability.  Immersive  MGUI  design  appears  based  largely  on  the  assumption  that  users 
who  are  immersed  in  such  an  environment  will  feel  a greater  sense  of  believability  and  involvement.  This,  in  turn,  may 
lead  to  more  active  exploration  and  enhanced  understanding. 
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Figure  9:  Initial  entry  screen  before  Figure  10:  Second  image  in  entry 

user  clicks.  transition  screen. 


Figure  11:  Main  screen  for  a sample  immersive  MGUI. 

It  is  quite  difficult  to  portray  an  Immersive  MGUI  in  still  pictures.  The  interactive  nature  of  the  program  and  the 
advantages  and  impressions  of  animation  and  zooms  are  lost  in  still  pictures.  This  paper  will  do  its  best  to  illustrate  how 
our  exemplary  history  lesson  might  be  implemented  in  an  Immersive  MGUI.  Figures  9 and  10  illustrate  the  transition 
sequence  that  takes  a user  from  initial  click  to  the  cemetery’s  main  screen. 

At  the  end  of  the  entry  transition  sequence,  users  will  find  themselves  at  the  main  screen  (see  figure  11).  Notice  that  the 
labeled  icon  approach  of  the  sample  Thematic  MGUI  (figure  4)  has  been  replaced  by  unlabeled  objects.  Notice  also  that 
there  are  many  more  objects  visible  on  this  screen.  Recognize  that  this  screen  is  actually  a window  on  a larger  “virtual” 
cemetery.  That  is,  when  learners  move  to  the  far  right  or  far  left  of  this  screen,  they  can  change  viewpoint  to  f ee  more 
cemetery  and  more  gravesites.  As  suggested  earlier  and  illustrated  in  figures  5-8,  the  caretaker  house  will  remain  a 
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centrally  accessible  device.  Thus,  navigation  within  this  environment  is  actually  rotation  around  the  hub  of  the  caretaker 
house. 

The  plot  map  in  the  caretaker’s  house  is  now  a reference  sheet,  not  a navigational  device.  Users  can  get  a copy  of  the  plot 
map  to  take  with  them  if  they  wish.  Notice  that  some  gravesites  appear  covered  with  snow.  Users  will  be  able  to  obtain 
devices  (brooms,  shovels,  rakes,  and  the  like)  at  the  caretaker’s  house  and  can  use  these  to  clear  gravesites.  Notice  also 
that  there  is  a fresh  gravesite  on  the  right  side  of  the  initial  screen.  Immersive  MGUIs  can  be  incredibly  rich 
environments,  providing  the  potential  for  a greater  sense  of  user  engagement  and  involvement. 

At  the  same  time,  however,  immeisive  metaphorical  GUIs  aren’t  for  everyone  or  every  application.  They  are 
exponentially  more  difficult  to  code.  The  task  at  hand  must  lend  itself  to  manipulation,  exploration,  and  three- 
dimensionality.  Lots  of  reading  isn’t  well  served  by  immersion.  One  has  to  consider  how  navigation  and  manipulation 
will  be  handled.  In  terms  of  navigation,  will  you  use  arrows  to  tell  the  user  that  he  or  she  can  go  right,  left,  up,  and 
down?  Will  you  allow  the  movement  of  the  mouse  pointer  to  cause  the  visible  screen  window  to  pan  across  some  larger 
virtual  screen?  If  so,  how  will  you  handle  the  loss  of  reference  point  so  famous  for  inducing  seasickness?  In  terms  of 
manipulation,  will  the  mouse  pointer  change  shape  or  form  when  different  actions  are  possible?  Will  you  employ 
sounds  to  assist  users  in  determining  what  is  happening?  In  short,  how  will  you  help  users  handle  the  disorientation  and 
confusion  that  often  accompany  more  complex  environments?  (Marchionini,  1988;  Oren,  1990).  In  order  for  Immersive 
MGUIs  to  function  effectively,  they  must  enhance  and  complement,  not  confuse  and  frustrate  users. 

How  will  you  help  users  maintain  continuity  and  context?  Will  you  supply  them  with  a backpack  or  a bag  in  which  to 
carry  useful  devices?  Will  you  supply  a notebook?  How  will  it  be  used  and  when?  Will  all  the  devices  the  user  carries 
be  accessible  all  of  the  time?  How  will  you  let  users  know  what  they  are  supposed  to  do  while  in  the  environment? 

How  will  you  determine  if  the  user  is  doing  what  he  or  she  needs  to  do  and  how  will  you  assess  the  user’s  progress 
toward  the  desired  goal  (if  such  a goal  exists)?  In  short,  there  are  many  crucial  design  considerations  that  require 
thoughtful  analysis  and  careful  design. 

Closing  Comments 

For  purposes  of  illustration,  this  paper  has  used  rather  simplistic  illustrations.  In  contrast,  the  demands  of  functionality 
and  content  may  well  dictate  that  a program  provide  substantially  more  of  both.  At  the  same  time,  this  paper  has 
attempted  to  highlight  what  might  be  considered  the  more  salient  concepts  and  questions  in  designing  metaphorical 
graphical  user  interfaces. 

Good  interface  design  goes  well  beyond  simple  adherence  to  rules  or  selection  of  pleasing  graphics.  It  addresses  user  needs 
and  the  demands  of  the  setting  in  which  the  program  will  be  used.  Thus,  interface  design,  like  all  good  software  design,  is 
a creative  act.  It  requires  great  care  and  great  forethought.  Since  metaphorical  interfaces  function  largely  by  tapping  into 
the  schemata  and  experiences  of  their  users,  designers  need  to  pay  particular  attention  to  the  models  and  experiences  of 
those  users.  Although  not  without  price,  a workable  and  useful  metaphor  is  a pearl  of  great  value. 
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Factors  Contributing  to  Students'  Preconceptions  of  Media 

Computers  are  becoming  an  integral  part  of  teaching;  yet  books  remain  the  primary  source  of  conveying 
instructional  content.  Television  and  videotapes  also  are  used  to  provide  realistic  pictures,  moving  images,  audio 
narration  and  music  to  supplement  instruction.  And  multimedia  provides  the  opportunity  to  combine  features  of  each  of 
these  three  media. 

Although  the  commonly  accepted  viewpoint  is  that  "...media  are  mere  vehicles  that  deliver  instruction  but  do 
not  influence  student  achievement  any  more  than  the  truck  that  delivers  our  groceries  causes  change  in  our  nutrition" 
(Clark,  1983,  p.  445),  Kozma  (1991)  has  challenged  and  expanded  this  commonly  held  view  and  proposed  that  the 
"capabilities  of  a particular  medium,  in  conjunction  with  methods  that  take  advantage  of  these  capabilities,  interact  with 
and  influence  the  ways  learners  represent  and  process  information  and  may  result  in  more  or  different  learning  when  one 
medium  is  compared  to  another  for  certain  learners  and  tasks  " (p.  179). 

Kozma  (1991)  defines  media  in  terms  of  it's  technology,  symbol  systems,  and  processing  capabilities.  The 
technology,  or  hardware,  primarily  affects  the  location  at  which  instruction  can  occur.  The  symbol  systems  of  a medium 
include  the  way  it  presents  verbal  and  visual  information.  Processing  capabilities  refer  to  the  ability  of  a medium  to 
provide  random  access  to  information,  learner  control  over  the  pacing,  and  other  arithmetic  and  logic  functions. 

Although  the  technology  is  assumed  to  have  minimal  effects  on  learning,  the  symbol  systems  and  processing  capabilities 
of  media  may  affect  the  nature  of  the  information  gained  and  the  way  the  information  is  processed.  Kozma  further  states 
that  "whether  or  not  a medium’s  capabilities  make  a difference  in  learning  depends  on  how  they  correspond  to  the 
particular  learning  situation-  the  tasks  and  learners  involved  [italics  added)-  and  the  way  the  medium’s  capabilities  are  used 
by  the  instructional  design"  (Kozma,  1991,  p.  182. 

Research  indicates  that  learners’  beliefs  about  an  instructional  delivery  system  may  be  one  factor  that  affects  the 
cognitive  processing  of  content  delivered  through  that  medium  (Krendl,  1986;  Richey,  1992;  Salomon,  1983;  Salomon 
& Leigh,  1984,  and  others).  Research  conducted  with  sixth  grade  students  found  that  the  perceived  ease  of  learning  from 
a medium  was  negatively  correlated  with  achievement  (Salomon,  1983;  Salomon  & Leigh,  1984);  however,  research 
conducted  with  post-secondary  learners  has  found  a significant  positive  correlation  between  the  perceived  ease  of  learning 
from  a medium  and  achievement  scores  (Cennamo,  Savenye,  & Smi,th,  1991;  Richey,  1992).  For  example,  Richey 
(1991 ; 1992)  used  path  analysis  to  examine  the  causal  influence  of  a number  of  variables  on  the  achievement  scores  of 
over  600  participants  in  industrial  training  programs.  She  found  that  "on  the  average,  sixty  percent  of  the  outcomes  of 
these  training  programs  can  be  predicted  by  the  adult  learners’  entering  characteristics  and  perceptions,  most  of  which  are 
directly  or  indirectly  related  to  one's  perceptions  of  the  training  delivery  system"  (Richey,  1 99 1 , p.  1 6).  These  results 
suggest  experienced  learners  may  be  accurately  aware  of  the  ease  with  which  they  learn  from  a particular  medium. 

But  what  influences  learners'  attitudes  toward  media?  Why  do  learners  view  one  medium  as  easier  or  more 
difficult  than  another  for  a particular  task?  Recognizing  the  learners’  voices  in  what  makes  it  easy  to  learn  a particular 
task  from  a particular  medium  may  provide  instructors  and  materials  developers  with  important  practical  implications  for 
their  design  and  development  efforts. 

Earlier  research  (Salomon,  1984;  Salomon  & Leigh,  1984;  Krendl,  1986)  found  that  learners'  preconceptions  of 
the  ease  or  difficulty  of  learning  a lesson  depended  on  the  medium  of  presentation;  however,  other  researchers  have  found 
that  the  perceived  ease  of  learning  a lesson  from  a particular  medium  was  also  dependent  upon  the  topic  of  the  lesson 
(Beentjes  ,1989)  and  the  learning  domain  (verbal,  intellectual,  attitude,  or  psychomotor)  of  the  instructional  objective 
(Cennamo,  1993b). 

Recently  a series  of  studies  has  been  conducted  to  determine  factors  influencing  learners'  perceptions  of  the  ease 
of  learning  a particular  task  using  a particular  medium.  In  an  exploratory  study  (Cennamo,  1993a),  12  preservice  teachers 
completed  a self-report  questionnaire  assessing  their  preconceptions  of  the  ease  of  achieving  various  learning  outcomes 
(psychomotor,  affective,  verbal,  intellectual)  using  the  media  of  interactive  video,  computers,  books,  and  television. 

They  were  interviewed  to  determine  the  reasoning  behind  their  ratings.  Analysis  of  the  interview  responses  indicated  that 
for  different  domains  of  learning  outcomes,  learners  used  different  criteria  for  rating  a medium  as  easy  or  difficult. 
Consistent  with  Kozma's  (1991)  theory,  learners'  ratings  of  the  ease  of  learning  a particular  task  using  a particular 
medium  were  influenced  by  the  a)  capabilities  of  the  medium  (such  as  the  symbol  systems  and  processing  capabilities), 
b)  characteristics  of  the  learner  (such  as  past  experience  and  personal  preferences),  and  c)  characteristics  of  the  task  (such 
as  complexity  of  the  learning  objective).  However,  the  particular  factors  that  were  perceived  to  be  of  major  importance 
varied  dej  nding  on  the  medium  and  domain  of  learning  outcome. 

The  current  study  attempted  to  validate  and  extend  the  prior  study.  The  purpose  of  this  study  was  to  determine 
whether  similar  patterns  are  found  a)  when  interviewing  a larger  number  of  participants  and  b)  when  focusing  on  a 
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specific  content  area.  Whereas  interviews  were  conducted  with  only  12  participants  in  the  previous  study,  38  interviews 
were  conducted  in  the  current  study.  In  addition,  the  questionnaire  used  to  elicit  verbal  responses  in  the  previous  study 
contained  learning  objectives  from  a variety  of  content  areas.  In  the  current  study,  the  interview  questions  have  been 
restricted  to  objectives  from  the  science  content  area. 

Methods 


Participants 

Thirty-eight  undergraduate  students  enrolled  in  an  undergraduate  computer  education  class  in  a large  midwestem 
university  volunteered  to  participate  in  individual  interviews.  Of  the  38  participants,  34  were  female  and  4 were  male. 
Twenty-four  (63%)  were  Education  majors,  1 1 (29%)  were  majoring  in  Child  Development  and  Family  Services,  and  three 
(8%)  were  majoring  in  other  areas.  42%  were  seniors  (16),  45%  were  juniors  (17),  and  13%  (5)  were  sophomores. 

Materials 

During  the  interview,  a preconceptions  questionnaire  was  used  to  evoke  students*  perceptions  of  the  ease  of  learning 
from  books,  television,  computers,  and  interactive  video.  It  included  a cover  sheet  that  operationally  defined  each  medium  in 
order  to  ensure  that  all  participants  interpreted  the  terms  “books”,  “television”,  “computers**,  and  “interactive  video**  in  the 
same  manner  (see  Appendix).  The  students  were  asked  to  rate  the  difficulty  of  learning  12  tasks  using  each  of  the  four  media 
on  a five-point  Likert  scale  and  to  orally  justify  their  ratings.  One  represented  “very  easy**  and  five  represented  “very 
difficult”.  The  questions  were  similar  in  format  to  those  used  by  Beentjes  (1989),  Salomon  (1984),  and  Cennamo  (1993a; 
1993b).  This  questionnaire  included  questions  such  as: 

- How  easy  would  it  be  to  learn  to  weigh  a sample  to  the  nearest  1/10  of  a gram  using  a digital  scale  from 


a book? 

very  easy 

1 

2 

3 

4 

5 

very  difficult 

television? 

very  easy 

1 

2 

3 

4 

5 

very  difficult 

a computer? 

very  easy 

1 

2 

3 

4 

5 

very  difficult 

interactive  video? 

very  easy 

1 

2 

3 

4 

5 

very  difficult 

There  were  three  questions  for  each  learning  domain  (intellectual,  verbal,  psychomotor,  and  attitudes).  The 
following  questions  stems  were  included: 

Verbal  information 

- How  easy  would  it  be  to  learn  to  list  three  rules  you  should  follow  when  working  in  a chemistry  lab  from... 

- How  easy  would  it  be  to  learn  to  define  terms  that  describe  the  basic  units  of  matter  including  atoms. 

elements,  molecules  and  compounds  from... 

- How  easy  would  it  be  to  learn  to  label  the  parts  of  a cell  from... 

Intellectual  Skills 

- How  easy  would  it  be  to  learn  to  balance  a chemical  equation  from... 

- How  easy  would  it  be  to  learn  to  determine  the  results  of  a genetic  cross  using  a Punnett  square  from... 

- How  easy  would  it  be  to  learn  to  develop  and  test  a hypothesis  when  conducting  an  experimental  study  from... 

Psychomotor  Skills 

- How  easy  would  it  be  to  learn  to  measure  a liquid  sample  usings  graduated  cylinder  from... 

- How  easy  would  it  be  to  learn  to  weigh  a sample  to  the  nearest  1/JQ  of  a gram  using  a digital  scale  from... 

- How  easy  would  it  be  to  learn  to  mount  and  correctly  focus  a slide  on  a microscope  from... 


Attitude  Skills 

- How  easy  would  it  be  to  learn  to  voluntarily  choose  to  wear  safety  glasses  while  working  a in  a chemistry  Jah 

from... 

- How  easy  would  it  be  to  learn  to  appreciate  the  biological  diversity  of  the  natural  world  from... 

- How  easy  would  it  be  to  learn  to  choose  to  treat  the  environment  in  a respectful  manner  from... 


BEST  COPY  AVAILABLE 
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Numerical  responses  were  averaged  across  domain  and  medium  to  provide  an  average  score  for  each  medium 
/domain  combination.  For  example,  there  was  an  average  score  for  books/  verbal,  books/  intellectual,  books/  attitudes, 
and  books/  psychomotor.  Chronbach's  alpha  yielded  an  acceptable  overall  reliability  coefficient  of  .79. 


Procedures 

During  the  seventh  week  of  class,  the  researcher  invited  the  participants  to  sign-up  for  individual  interviews. 
Although  the  participants  were  offered  a small  amount  of  extra  credit  for  participation  in  the  interview,  the  structure  of 
the  course  allowed  the  students  to  earn  course  credit  in  a variety  of  ways  throughout  the  semester.  As  they  reported  to  the 
interview  site,  a research  assistant  explained  the  purpose  of  the  interview  and  presented  the  participants  with  the 
questionnaire.  Borg  and  Gall  (1993)  indicate  that  matching  interviewers  and  respondents  on  variables  such  as  social  class, 
age,  and  gender  may  produce  more  valid  responses;  thus,  like  the  majority  of  the  respondents,  the  interviewer  was  an 
undergraduate  student,  female,  and  approximately  20  years  old. 

Using  an  interview  guide,  she  conducted  each  interview  in  a consistent  manner.  The  interviewer  read  the  cover 
sheet  aloud  and  waited  for  the  respondents  to.complete  the  demographic  questions.  Then  the  interviewer  read  each 
question  aloud.  The  students  were  asked  to  indicate  their  rating  on  the  5-point  scale  and  to  orally  reflect  on  why  they 
rated  the  medium  as  they  did.  These  conversations  were  audiotaped  and  later  transcribed. 

Data  Analysis  and  Results 

Initial  analysis  focused  on  student’s  familiarity  with  the  four  media.  One-hundred  percent  of  the  respondents 
indicated  that  they  used  books  once  a month  or  more,  97%  indicated  that  they  watched  television  once  a month  or  more, 
and  92%  indicated  that  they  used  computers  once  a month  or  more.  However,  92%  of  the  respondents  indicated  that  they 
"hardly  ever"  used  interactive  video.  Based  on  tire  infrequency  of  use,  responses  to  questions  addressing  the  perceived  ease 
of  learning  from  interactive  video  were  eliminated  from  the  data  analysis.  Subsequent  analysis  focused  on  learners’ 
perceptions  of  the  perceived  ease  of  learning  science  content  from  books,  television,  and  computers. 

The  interview  responses  were  of  primary  importance  in  determining  factors  that  contribute  to  learners' 
perceptions  of  the  ease  of  learning  from  a particular  medium.  Responses  to  the  questionnaire  were  analyzed  quantitatively 
to  provide  guidance  in  interpreting  the  interview  responses.  Student  ratings  on  the  Likert  scale  were  analyzed  using  a 
Repeated  Measures  Analysis  of  Variance  to  determine  whether  students  preconceptions  of  the  ease  of  learning  science 
tasks  varied  by  learning  domain,  by  medium,  or  randomly  depending  on  topic.  Based  on  the  findings  of  the  statistical 
analysis,  the  interview  data  were  analyzed  to  determine  factors  influencing  learners'  preconceptions  of  the  ease  of  learning 
a skill  in  a given  domain  using  a particular  medium. 

Analysis  of  Questionnaire  P Us 

The  Repeated  Measures  Analysis  of  Variance  indicated  several  significant  findings  concerning  learners' 
preconceptions  of  the  ease  of  learning  a given  outcome  using  a given  medium  of  presentation.  (See  Table  1 .)  Following 
the  finding  of  a significant  f,  Tukey's  Honestly  significant  Difference  (HSD)  test  was  used  to  determine  significant 
differences  among  individual  group  means. 

As  expected,  there  was  a significant  interaction  among  learning  domain  and  medium,  F(l,37)  = 29.04,  p<  .001. 
Learners  perceived  that  it  was  significantly  easier  to  learn  psychomotor  skills  from  television  (M  = 2. 10)  than  from 
computers  (M  = 2.75)  and  books  (M  = 3.28),  and  significantly  easier  to  learn  psychomotor  skills  from  computers  than 
from  books.  For  the  learning  of  attitudes,  learners  also  perceived  it  to  be  significantly  easier  to  learn  from  television  (M 
= 1 .48)  than  from  books  (M  = 2.71)  and  computers  (M  = 2.93).  Learners  perceived  it  to  be  significantly  easier  to  learn 
intellectual  skills  from  computers  (M  = 2.27)  and  television  (M  = 2.63)  than  from  books  (M.  = 3.00).  Learners  rated 
computers  (M  = 1 .70)  and  books  (M  = 1 -84)  as  significantly  easier  than  television  (M  = 2.29)  for  learning  verbal 
information.  As  expected,  there  was  a significant  difference  among  media,  F (1,37)  =18.98,  p<  001 , and  among  learning 
domains,  F(l,37)  = 47.01,  pc.001.  Tukey’s  post-hoc  analysis  of  the  data  revealed  that  television  (M  = 2. 13)  was 
perceived  to  be  significantly  easier  than  computers  (M  = 2.42)  and  books  (M  = 2.7 1 ),  and  computers  were  perceived  to 
be  significantly  easier  than  books.  Students  perceived  that  It  was  significantly  easier  to  learn  verbal  information  (M  = 

1 .85)  than  attitudes  (M  = 2.15),  intellectual  skills  (M  = 2.42)  and  psychomotor  .skills.(M  = 2.44),  and  significantly  easier 
to  learn  attitudes  than  psychomotor  skills. 
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Table  1:  Means  and 

standard  deviations 

Computers 

of  ratings  on 

Television 

preconceptions  questionnaire. 

Books  TOTAL  BY  DOMAIN 

Psychomotor 

Mean 

2.75 

2.10 

3.28 

2.44 

SD 

1.04 

1.03 

1.12 

1.16 

Attitudes 

Mean 

2.93 

1.48 

2.71 

2.15 

SD 

1.09 

.74 

1.15 

1.16 

Verbal 

Mean 

1.70 

2.29 

1.84 

1.85 

SD 

.86 

1.14 

.95 

.99 

Intellectual 

Mean 

2.27 

2.63 

3.00 

2.42 

SD 

.99 

1.57 

1.21 

1.15 

TOTAL  BY 

Mean 

2.42 

2.13 

2.71 

MEDIUM 

SD 

1.10 

1.11 

1.23 

N(?te:  Higher  scores  mean  greater  perceived  difficulty. 
N=38. 

Intervirw  analysis  and  results 

Based  on  the  results  of  the  statistical  analysis,  the  interview  data  was  interpreted  in  terms  of  the  media  by 
domain  interaction.  Data  from  the  interviews  were  entered  into  a database  and  coded  as  to  the  question  number,  medium, 
and  domain  addressed  by  each  response.  The  data  were  sorted  by  medium  and  domain,  resulting  in  a set  of  responses  for 
books/ verbal,  books/  intellectual,  books/  attitudes,  books/  psychomotor,  television/verbal,  television/  intellectual, 
television/  attitudes,  television/  psychomotor,  computers/verbal,  computers/  intellectual,  computers/  attitudes,  and 
computers/  psychomotor. 

Two  researchers  independently  read  each  statement  associated  with  each  medium/domain  combination  and  tallied 
responses  that  were  similar  in  meaning.  For  example  ’’can  go  back”,  ’’can  select  what  you  want  to  study”,  and  ’’could 
skip  parts  you  knew”  were  all  coded  as  ’’Learner  Control”. 

Based  on  the  categories  that  emerged  during  the  previous  exploratory  study  (Cennamo,  1993a),  the  list  of 
responses  was  clustered  into  the  following  categories:  symbol  system,  processing  capabilities,  task  characteristics, 
personal  preferences,  and  characteristics  of  the  technology.  For  example  ’’learner  control”,  "practice  and  feedback”,  and 
"lack  of  interaction"  all  refer  to  the  processing  capabilities  of  the  medium.  After  clustering  the  responses  by  category, 
the  responses  in  each  category  were  reviewed  to  verify  the  placement  of  each  response. 

Responses  in  each  category  were  examined  once  again  to  determine  if  the  responses  could  be  further  clustered 
within  each  category.  This  analysis  revealed  that  responses  addressing  the  symbol  systems  presented  by  the  various 
media  primarily  were  concerned  with  the  pictures  present,  the  way  the  verbal  information  was  presented,  the  ability  of  the 
medium  to  present  demonstrations,  the  examples  and  reasons  provided,  the  ability  to  highlight  important  information, 
miscellaneous  negative  characteristics  of  the  medium,  and  other  miscellaneous  responses.  Responses  that  addressed  the 
processing  capabilities  of  the  medium  were  primarily  concerned  with  the  ability  of  the  medium  to  provide  practice  and 
feedback,  learner  control,  the  ability  of  the  student  to  ask  questions  or  clarify  understandings,  and  other  miscellaneous 
responses.  Responses  coded  as  personal  preferences  were  mostly  individualistic  in  nature;  however,  clusters  that  emerged 
were  based  on  learners’  past  experiences  or  reflected  their  dislike  for  the  media.  Task  characteristics  clustered  into  those 
that  addressed  the  ease  or  difficulty  of  the  task,  the  suitability  of  the  medium  for  the  task,  and  other  miscellaneous 
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responses.  Responses  that  mentioned  characteristics  of  the  technology  were  rare,  thus,  the  technology  category  was  not 
subdivided  further. 

Typically,  learning  outcomes  are  determined  prior  to  selecting  media  to  deliver  the  content,  thus,  the  results  of 
this  analysis  will  be  discussed  by  learning  domain  of  the  intended  outcome. 

Attitude  skills 

Television  was  rated  as  significantly  easier  than  computers  and  books  for  learning  attitude  skills  (see  Figure  1 
and  Appendix  B).  The  ability  of  a medium  to  demonstrate  the  target  skill  using  realistic  pictures  seemed  to  be  the  most 
prevalent  characteristics  influencing  learners  ratings  of  the  ease  of  learning  attitudes  using  the  three  media.  In  general, 
pictures  were  important  for  the  learning  of  attitudes.  The  realistic  nature  of  the  pictures  influenced  learners'  ratings  of  the 
ease  of  leaning  an  attitude  skill.  The  presence  of  a variety  of  visual  examples  on  television  also  was  important  to  the 
participants  of  the  study. 

Figure  1.  Mean  ratings  of  the  ease  or  difficulty  of  learning  attitude  skills. 

. Easiest Most  Difficult 

Means  1.48  a 2.71  b 2.93  b 

Television  Books  Computers 

Note:  means  with  different  subscripts  are  significantly  different  from  one  another. 


The  respondents  felt  that  when  using  computers  they  would  be  primarily  "just  reading"  "just  facts"  and  that 
words  would  not  be  very  valuable  in  changing  their  attitudes.  Respondents  also  perceived  that  books  mil  you 
information  and  present  facts,  but  mentioned  that  they  were  "just  words"  and  indicated  that  some  ideas  could  not  be 
described  well  in  words.  They  frequently  mentioned  that  books  did  not  present  real  situations  and  that  you  would  not  see 
the  results  of  not  changing  your  attitude.  However,  the  respondents  did  acknowledge  that  books  an  computers  could  list 
the  possible  results  of  an  action  and  the  consequences  or  reasons  why  one  should  change  an  attitude.  Participants 
acknowledged  that  television  also  would  present  verbal  information  with  respondents  mentioning  the  audio  track  as 
important  for  providing  verbal  descriptions. 

The  processing  capabilities,  characteristics  of  the  task,  personal  preferences,  and  technology  of  a medium  did  not 
appear  to  be  as  influential  as  the  symbol  systems  in  learners'  perceptions  of  the  ease  of  learning  an  attitude  skill. 

Personal  preferences  were  stronger  for  books  than  the  other  two  media;  however,  most  ideas  were  mentioned  only  once, 
reflecting  the  individual  nature  of  personal  preferences.  The  perception  that  reading  a book  would  not  influence  personal 
actions  was  the  most  frequently  mentioned  idea  coded  as  a personal  preference. 

Intellectual  Skills 

Although  computers  were  rated  as  slightly  easier  than  television  for  the  learning  of  intellectual  skills,  the 
difference  was  not  significant;  both  media  were  rated  as  significantly  easier  than  books  in  the  learning  of  intellectual 
skills  (see  Figure  2 and  Appendix  C).  Although  television  and  computers  were  rated  similarly  for  learning  intellectual 
skills,  the  ratings  were  attributed  to  very  different  characteristics  of  the  media. 


Figure  2.  Mean  ratings  of  the  ease  or  difficulty  of  learning  intellectual  skills. 

Easiest Most  Difficult 

Means  2.27  a 2.63  a 3.00  b 

Computers  Television  Books 

Note:  means  with  different  subscripts  are  significantly  different  from  one  anothe > 


Participants  perceived  that  demonstrations  facilitated  the  ease  of  learning  intellectual  skills.  Once  again,  the 
strength  of  television  seemed  to  be  in  its  ability  to  demonstrate  the  target  skill  in  a step-by  step  manner.  These 
characteristics  were  further  enhanced  by  the  audio  capacities  of  television  that  allow  the  program  to  "tell",  teach,  and 
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explain  information.  The  ability  of  computers  to  show  how  a tasks  is  performed  and  "take  you  through  it,  step-by-step" 
also  was  an  important  characteristic  influencing  participants’  ratings  of  computers.  These  characteristics,  coupled  with 
the  textual  data  and  instructions  contributed  to  the  perceived  ease  of  learning  intellectual  skills  from  computers.  The 
perceived  textual  nature  of  computer  demonstrations  may  have  accounted  for  the  fairly  similar  number  of  responses 
mentioning  step-by-step  explanations  in  books.  The  strength  of  books  seemed  to  be  the  presence  of  examples  that 
illustrate  the  target  skill,  which  may  reinforce  the  learners  perceived  need  for  demonstrations  of  the  target  skill. 

The  processing  capabilities  of  computers  were  viewed  as  important  factors  contributing  to  the  perceived  ease  of 
learning  intellectual  skills.  The  presence  of  practice  and  feedback  was  the  characteristic  most  responsible  for  the  higher 
rating  of  computers  than  of  books  and  television.  Learner  control  was  also  perceived  as  critical  in  learning  intellectual 
skills.  The  most  frequently  mentioned  feature  of  learner  control  was  the  ability  to  review  and  repeat  information. 

Task  characteristics,  personal  preferences,  and  the  technology  had  little  influence  on  the  perceived  ease  of 
learning  intellectual  skills.  The  most  frequently  mentioned  task  characteristic  was  the  appropriateness  of  the  medium  for 
teaching  the  task.  Once  again,  personal  preferences  inflrenced  students’  ratings  of  books  more  frequently  than  the  other 
media.  The  frequency  of  responses  mentioning  characteristics  of  the  technology  as  influential  in  their  ratings  was  similar 
across  media. 


Television  was  rated  as  the  easiest  medium  from  which  to  learn  psychomotor  skills  (see  Figure  3 and  Appendix 
D).  The  presence  of  demonstrations  was  especially  influential  in  the  learners’  rating  of  the  media.  The  ability  of  a 
medium  to  present  directions  and  "tell  how"  in  a step-by-step  manner  also  was  considered  in  the  learners’  ratings  of  the 
ease  of  learning  psychomotor  skills  from  the  three  media. 


Figure  3.  Mean  ratings  of  the  ease  or  difficulty  of  learning  psychomotor  skills. 


Easiest Most  Difficult 

Means  2.10  a 2.75  b 3.28  c 

Television  Computers  Books 


Note:  means  with  different  subscripts  are  significantly  different  from  one  another 


Lack  of  practice  opportunities  was  perceived  as  a disadvantage  when  learning  psychomotor  skills  from  television 
and  books,  while  respondents  did  not  perceive  the  processing  capabilities  of  computers  as  particularly  relevant  to  learning 
psychomotor  skills.  Students  mentioned  that  they  would  not  be  "actually  doing  it”  when  learning  psychomotor  skills 
on  the  computer. 

The  perceived  ease  or  difficulty  of  the  task  accounted  for  the  majority  of  the  "task  related"  responses.  However, 
the  suitability  of  the  task  for  presentation  via  computer  was  the  most  frequently  mentioned  "task  variable”  influencing 
students’  ratings  of  computers. 

Personal  preferences  and  the  technology  were  not  particularly  influential  in  students'  ratings  of  the  ease  of 
learning  psychomotor  skills 

Verbal  information 

Students  perceived  it  to  be  easier  to  learn  verbal  information  from  computers  and  books  than  from  television. 
(See  Figure  4 and  Appendix  E.)  Surprisingly,  the  presence  of  pictures  illustrations,  and  diagrams  were  perceived  as 
important  features  influencing  the  ease  of  learning  verbal  information  from  all  three  media.  Students  continued  to  value 
the  demonstrations  provided  by  television.  Also  surprisingly,  students  perceived  the  three  media  as  being  fairly  equal  in 
their  ability  to  present  verbal  information. 

Figure  4.  Mean  ratings  of  the  ease  or  difficulty  of  learning  verbal  information. 

Easiest Most  Difficult 

Means  1.70  a 1.84  a 2.29  b 

Computers  books  Television 

Hots:  means  with  different  subscripts  are  significantly  different  from  one  another 
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The  processing  capabilities  of  the  media  were  important  to  the  learners'  ratings  of  the  ease  of  learning  verbal 
information  from  books,  computers,  and  television.  Participants  mentioned  the  ability  to  review  and  refer  back  to 
information  presented  through  books  and  computers.  This  ability  to  refer  back  to  information  with  ease,  coupled  with 
the  participants  perceptions  that  the  task  required  memorization  and  rehearsal  of  the  information  may  have  accounted  for 
the  perceived  ease  of  learning  verbal  information  from  computers  and  books.  The  ability  of  the  computer  to  provide  drill 
and  practice  with  feedback  also  contributed  to  the  perceived  ease  of  learning  verbal  information  from  computers. 

Participants  felt  that  the  ease  or  difficulty  of  the  task  was  more  relevant  to  learning  from  books  than  other 
media.  In  addition,  students  perceived  books  to  be  more  appropriate  for  learning  verbal  information  than  computers  or 
television. 

Neither  personal  preference  or  the  technology  seemed  to  be  particularly  important  to  the  perceived  ease  of 
learning  verbal  information  from  the  three  media. 

Summary  and  Discussion 

Discussion 

This  study  was  conducted  as  an  initial  step  in  determining  factors  that  contribute  to  learners'  preconceptions  of 
the  ease  or  difficulty  of  learning  science  tasks  in  the  attitude,  psychomotor,  intellectual,  and  verbal  information  domains 
using  computers,  television,  and  books. 

Consistent  with  the  results  of  a previous  study  (Cennamo,  1993a),  the  students’  ratings  of  the  ease  of  learning 
various  tasks  using  computers,  television,  and  books  seemed  to  reflect  some  awareness  of  the  external  conditions  needed 
for  optimal  learning  in  each  domain  (Gagnd,  1985;  Gagn<5,  Briggs,  & Wagner,  1992).  Opportunities  for  rehearsal  can 
enhance  the  learning  of  verbal  information,  and  books  and  computers,  the  two  media  where  the  pacing  is  under  the 
control  of  the  learner,  were  rated  as  the  easiest  from  which  to  learn  verbal  information.  Likewise,  the  learners  ratings  of 
the  ease  of  learning  intellectual  skills  from  computers  reflected  an  awareness  that  opportunities  for  practice  and  feedback 
with  a variety  of  examples  can  enhance  the  learning  of  intellectual  skills. 

From  the  responses  to  the  interview  questions,  it  appears  that  for  these  non-science  majors,  visual  imagery  is 
important  in  learning  science  content.  Learners  in  a prior  study  (Cennamo,  1993a)  who  responded  to  questions  about  the 
ease  of  learning  a variety  of  skills  in  several  content  areas  indicated  that  demonstrations  were  important  in  learning 
psychomotor  skills  and  attitudes;  the  learners  who  responded  to  questions  specifically  addressing  science  content  indicated 
that  demonstrations  enhanced  learning  for  all  domains  but  verbal  information.  However,  these  learners  indicated  that 
pictures  and  illustrations  increase  the  ease  of  learning  verbal  information.  Kozma  (1991)  indicates  that  visual  images  are 
particularly  important  when  learning  new  content,  suggesting  that  pictures  provide  learners  with  a mental  model  onto 
which  they  can  then  map  the  information  provided  by  the  text.  It  is  likely  that  the  undergraduate  students  majoring  in 
education  and  child  and  family  development  may  have  lacked  confidence  in  their  understanding  of  the  science  content, 
thus,  they  may  have  felt  the  need  for  a concrete  visual  image. 

Kozma  (1991)  suggested  that  "whether  or  not  a medium's  capabilities  make  a difference  in  learning  depends  on 
how  they  correspond  to  the  particular  learning  situation-  the  tasks  and  learners  involved-  and  the  way  the  medium’s 
capabilities  are  used  by  the  instructional  design.”  (p.  182).  From  the  responses  to  this  study,  it  seems  that  students' 
perceptions  of  the  ease  of  learning  a particular  skill  using  a particular  medium  was  influenced  by  their  perceptions  of  the 
way  a medium’s  characteristics  are  used  by  the  instructional  design.  They  perceive  television  to  include  demonstrations 
and  realistic  pictures.  They  perceive  computers  to  include  graphics  and  practice  with  feedback.  As  students  have  more 
experience  with  computer  programs  capable  of  presenting  visual  and  verbal  information  with  the  realism  currently 
associated  with  television,  students’  perceptions  of  computers  will  likely  change  to  reflect  a broader  definition  of 
computer  assisted  instruction.  In  addition,  they  indicated  that  "the  task  and  learners  involved"  did  influence  their  ratings 
of  the  ease  of  learning  science  skills  from  the  three  media.  Personal  preferences  and  characteristics  of  the  task  such  as  the 
domain  of  the  target  skill,  the  perceived  ease  or  difficulty  of  the  target  task,  and  the  perceived  appropriateness  of  the  media 
for  teaching  the  task  were  important  to  the  participants. 

These  results  reinforce  that  it  is  important  to  consider  the  target  skill  in  selecting  media  for  instruction.  Given 
the  results  of  this  study,  it  is  surprising  that  the  majority  of  college  level  instruction  uses  books  as  the  primary  method 
of  relaying  science  content.  Most  college  level  instruction  focus  on  teaching  intellectual  skills  such  as  concepts,  rules, 
and  problem  solving  rather  than  attitudes,  psychomotor  skills,  or  verbal  information.  However,  books  were  rated  as  the 
most  difficult  medium  from  which  to  learn  intellectual  skills.  Although  science  majors  may  have  different  perceptions  of 
the  ease  of  learning  from  books,  this  study  suggests  that  perhaps  computers  should  be  used  to  teach  intellectual  skills  and 


verbal  information  to  novice  learners.  Perhaps  television  or  videotapes  should  be  used  to  teach  attitudes  and  psychomotor 
skills.  Perhaps  it  will  be  even  better  if  designers  of  computer-based  lessons  for  teaching  science  content  incorporate 
"television  like"  sequences  and  images  in  multimedia  programs  designed  to  teach  intellectual,  attitude,  or  psychomotor 
skills.  However,  the  inclusion  of  realistic,  moving  images  and  sound  in  computer-based  instruction  requires  large 
amounts  of  computer  memory  and  storage.  Thus,  an  awareness  of  situations  where  learners  perceive  realistic  images  and 
sound  as  important  for  the  ease  of  learning  remains  important. 
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Appendix  A,  Covgr  sheet  for  preconceptions  questionnairet 


Preconceptions  Questionnaire  NUMBER 

This  questionnaire  is  designed  to  examine  your  preconceptions  of  four  forms  of  media.  Specifically,  it  is  designed  to 
determine  your  perceptions  of  the  ease  or  difficulty  of  learning  from  books,  television,  computers  and  interactive  video. 

As  you  know,  instruction  can  be  presented  in  a variety  of  media  formats.  Throughout  your  education,  you  have  probably 
read  many  books  and  seen  quite  a few  videotapes.  You  may  or  may  not  have  had  experience  in  computer  assisted  learning 
or  interactive  video  based  instruction.  Since  this  questionnaire  is  about  your  preconceptions  of  books,  television, 
computers,  and  interactive  video,  the  terms  will  be  operationally  defined  as  explained  below: 

For  the  purposes  of  this  questionnaire,  when  you  see... 

Books:  Please  imagine  that  you  are  reading  text  materials  supplemented  with  photographs  and  illustrations. 


Television:  Please  imagine  that  you  are  watching  a series  of  moving  video  images,  supplemented  with  narration, 
music  and  other  sound  effects.  Appropriate  illustrations,  still  pictures,  or  text  screens  may  also  be  presented. 

Computers:  Please  imagine  that  you  are  using  a computer  program  that  contains  text  screens  and  graphic  illustrations. 
Animation  may  be  used  to  create  moving  images.  There  is  no  sound.  At  certain  points  in  the  program,  it  will  be 
necessary  to  make  a selection  from  a menu  or  to  respond  to  a prompt  or  question  before  the  program  will  continue. 

Interactive  video:  Please  imagine  that  you  are  using  a program  that  combines  computer  generated  screens  with  video 
images.  The  video  display  may  include  moving  and  still  images,  narration,  music  and  sound  effects.  The  computer 
display  may  include  text  screens,  graphic  illustrations,  and  animation.  At  certain  points  in  the  program,  it  will  be 
necessary  to  make  a selection  from  a menu  or  to  respond  to  a prompt  or  question  before  the  program  will  continue. 


Please  indicate  your  gender ( ) male  ( ) female 


Please  indicate  how  frequently  you  use 


Books 


( ) almost  never  ( ) once  a year 


)once  a month  ( ) once  a week  or  more 


Television 

Computers 


) almost  never  ( ) once  a year 
( ) almost  never  ( ) once  a year 


( )once  a month  ( ) once  a week  or  more 
( )once  a month  ( ) once  a week  or  more 


Interactive 

Video  ( ) almost  never  ( ) once  a year  ( )once  a month  ( ) once  a week  or  more 
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Appendix  B.  Factors  influencing  participants’  ratings  of  the  ease  or  difficulty  of  learning  attitude  skills  from 
computers,  books  and  television. 
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Appendix  C Factors  influencing  participants'  ratings  of  the  ease  or  difficulty  of  learning  intellectual  skills  from 
computers,  books  and  television. 
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Appendix  D.  Factors  influencing  participants'  ratings  of  the  ease  or  difficulty  of  learning  psvchomotyi  skill?  from 
computers,  books  and  television. 
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Appendix  E.  Factors  influencing  participants'  ratings  of  the  ease  or  difficulty  of  learning  verbal  information  from 
computers,  books  and  television^ 
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The  Effects  of  Relevance  on  Mental  Effort 

It  is  a commonly  accepted  that  learners  view  video  as  an  easy,  passive  medium,  and  thus  invest  little  mental 
effort  in  processing  the  information  (the  "couch  potato").  This  assumption  is  primarily  based  on  a series  of  studies  that 
Salomon  (Salomon,  1983;  Salomon,  1984;  Salomon  & Leigh,  1984)  conducted  in  the  early  eighties.  In  these  studies, 
Salomon  found  that  learners  perceived  it  to  be  "easier"  to  learn  from  television  than  books,  reported  spending  less  mental 
effort  in  learning  from  videotape  than  from  books,  and  recalled  less  from  a video-based  version  of  a story  than  from  a 
print  -based  version.  Salomon  assumed  that  learners  perceptions  of  video  or  television  as  "easy"  resulted  in  the 
investment  of  less  mental  effort,  and  consequently  less  learning  achievement,  when  compared  to  reading  books. 

We  could  assume  from  these  conclusions  that  it  is  preferable  to  provide  instruction  in  a printed  form;  however, 
video  remains  a popular  means  of  conveying  instruction,  particularly  in  the  training  environment.  In  fact,  a recent 
survey  conducted  by  Training  magazine  reported  that  the  use  of  videotape  for  conducting  corporate  training  was  exceeded 

only  by  stand-up  instruction  (Froiland,  1993). 

The  majority  of  research  investigating  the  mental  effort  invested  in  instructional  videotapes  has  been  conducted 
with  elementary  and  middle  school  children  (Beentjes  & van  deer  Voort,  1993;  Sherman,  Salomon,  1983;  Salomon, 

1984;  Salomon  & Leigh,  1984).  Although  several  researchers  have  investigated  the  amount  of  effort  that  college 
students  invest  in  processing  print  lessons  (Britton,  Muth,  & Glynn,  1986;  Britton,  Piha,  Davis,  & Wehausen,  1978, 
and  others)  and  television  commercials  (Lang,  Geiger,  Strickwerda,  & Sumner,  1993;  Reeves  & Thorson,  1986,  and 
others),  there  has  been  very  little  research  that  has  systematically  investigated  the  mental  effort  older  learners  expend  in 
learning  from  video-based  instructional  materials.  As  older  learners  may  have  more  well  developed  strategies  for  learning 
from  videotape  that  elementary  and  middle  school  students,  it  is  important  to  investigate  the  extent  to  which  Salomon  s 
conclusions  apply  to  the  adult  population. 

Salomon  (1983a)  defines  the  construct  of  "mental  effort"  as  the  "number  of  non-automatic  elaborations  applied 
to  a unit  of  material"  (p.  42).  In  contrast  to  automatic  processing  that  is  fast  and  effortless,  non-automatic  processing  is 
deliberate,  conscious,  and  very  much  under  the  control  of  the  individual.  As  new  information  is  received,  the  new 
information  cues  the  retrieval  of  related  prior  knowledge  (E.  Gagne,  1985)  stored  in  the  learner  s schemata.  The  learner  is 
then  able  to  make  connections  between  the  new  information  and  information  retrieved  from  prior  knowledge;  this  process 
is  referred  to  as  elaboration  (E.  Gagne,  1985).  The  increased  contact  with  the  learner's  mental  schemata  that  results  from 
the  conscious,  non-automatic  generation  of  elaborations,  or  mental  effort,  is  presumed  to  facilitate  the  retention  and 
retrieval  of  the  new  material. 

The  search  for  techniques  to  increase  the  effort  that  learners  invest  in  video-based  instruction  has  been  hindered 
by  the  limitations  of  the  instruments  used  to  assess  the  construct  of  mental  effort.  Early  work  (Salomon,  1983; 

Salomon,  1984;  Salomon  & Leigh,  1984)  that  investigated  the  amount  of  effort  learners  invested  in  print  and  video-based 
instruction  used  self-report  questionnaires  to  document  the  learners'  effort  expenditures.  However  recently,  researchers 
(Beentjes,  1989;  Cennamo,  Savenye,  & Smith,  1991 ) have  identified  a need  for  more  precise  methods  of  assessing 
mental  effort.  Beentjes  (1989)  accurately  states  that  "...  validation  studies  in  which  mental  effort  is  assessed  by 
multiple  methods  are  called  for."  ( p.  56). 

Secondary  task  techniques  have  been  used  to  assess  effort  in  a variety  studies  using  print  (Britton,  Muth,  & 
Glynn,  1986;  Britton,  Piha,  Davis  & Wehausen,  1978;  and  others)  and  television  commercials  (Lang,  Geiger, 
Strickwerda,  & Sumner,  1993;  Reeves  & Thorson,  1986;  and  others),  and  seem  to  offer  a promising  method  of  assessing 
the  mental  effort  invested  in  video-based  lessons.  When  using  the  secondary  task  technique,  a participant  completes  a 
primary  task  (reading,  watching  a video,  etc.)  and  at  the  same  time  is  instructed  to  respond  to  a secondary  task  such  as  a 
beep  or  flash  of  light  as  quickly  as  possible.  Researchers  assume  that  as  learners  spend  more  cognitive  resources  on  the 
primary  task,  they  have  less  cognitive  resources  left  to  allocate  to  the  secondary  task.  They  assume  that  slower  reaction 
times  to  the  secondary  task  indicate  that  additional  cognitive  effort  is  being  allocated  to  the  primary  task. 

Although. Salomon's  questionnaire  and  the  secondary  task  technique  are  assumed  to  both  measure  mental  effort, 
the  results  of  a recent  study  call  that  assumption  into  question.  Beentjas  and  van  dcr  Voort  (1993)  assessed  the  effort  that 
fourth  and  sixth  grade  students  invested  in  two  print-based  stories  and  the  "structurally  equivalent"  video-based  versions. 
While  students  were  reading  or  viewing  the  stories,  they  responded  to  a secondary  task,  an  audible  beep,  by  pressing  a key 
as  quickly  as  possible  after  hearing  the  beep.  After  they  completed  the  stories,  they  responded  to  a self-report 
questionnaire  of  mental  effort  that  was  based  on  Salomon's  instrument.  Then  they  completed  an  achievement  test.  The 
researchers  found  very  low  correlations  between  students  scores  on  the  questionnaire  and  the  secondary  task  measures  of 
effort.  In  addition,  the  correlations  between  scores  on  both  measures  and  achievement  scores  on  a test  of  recall  inferences 
were  also  very  low.  To  further  complicate  the  search  for  a valid  means  of  assessing  mental  effort,  students  reported 
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spending  significantly  more  effort  in  reading  the  text  versions  than  in  viewing  the  videotapes.  However,  when  reaction 
times  to  the  secondary  task  were  used  as  a measure  of  mental  effort,  the  opposite  effects  were  found;  reaction  times  were 
significantly  longer,  indicating  more  mental  effort,  for  the  videotaped  versions  than  the  text  versions.  Beentjas  and  van 
der  Voort  state  that  further  research  is  needed  on  the  validity  of  both  methods  of  assessing  mental  effort.  They  suggest 
that  one  promising  area  of  investigation  may  be  to  determine  to  what  extent  both  measures  are  sensitive  to 
manipulations  that  are  intended  to  increase  mental  effort. 

But  what  manipulation?  No  prior  studies  have  manipulated  variables  designed  to  increase  mental  effort  and 
assessed  increases  in  effort  using  the  secondary  task  technique  and  a questionnaire.  To  select  a manipulation  that  has  been 
shown  to  increase  mental  effort  as  measured  by  a self-report  questionnaire  (fun/leam  for  example)  may  be  biased  toward 
demonstrating  effects  on  the  questionnaire.  To  select  a variable  that  has  been  shown  to  increase  mental  effort  as  measured 
by  the  secondary  task  technique  (related  or  unrelated  cuts,  for  example)  may  be  biased  toward  demonstrating  effects  on  the 
secondary  task  measure  . To  further  complicate  the  selection  of  variables,  the  vast  majority  of  research  that  has  used 
reaction  times  to  a secondary  task  as  a dependent  variable  has  manipulated  features  of  either  text  or  video,  not  both.  To 
select  a variable  that  is  effective  in  print  may  bias  the  results  toward  print;  conversely,  to  select  a variable  that  is  effective 
in  videotape  may  bias  the  results  toward  that  medium. 

Mental  effort  refers  to  the  choice  on  the  part  of  the  learner  to  allocate  cognitive  resources  to  elaborating  on  an 
idea,  thus,  it  has  motivational  as  well  as  cognitive  components  (Salomon,  1983).  According  to  Keller  (1983), 
motivation  "refers  to  the  choices  people  make  as  to  what  experiences  or  goals  they  will  approach  or  avoid,  and  the  degree 
of  effort  they  will  exert  in  that  respect."  (p.  389,  italics  in  original).  He  further  distinguishes  between  effort  and 
performance:  "Performance  means  actual  accomplishment,  whereas  effort  refers  to  whether  the  individual  is  engaged  in 
actions  aimed  at  accomplishing  the  task.  Thus  effort  is  a direct  indicator  of  motivation"  (p.  391).  Based  on  the  literature 
in  motivational  theory,  Kellers  ARCS  Model  of  Motivation  (1983)  breaks  the  concept  of  motivation  down  into  four 
parts  (attention,  relevance,  confidence  and  satisfaction).  Keller  provides  specific  strategies  that  instructional  designers  can 
use  to  increase  learner's  motivation,  or  the  effort  expended  in  accomplishing  an  instructional  task. 

According  to  Keller's  (1983)  ARCS  model,  one  promising  technique  for  increasing  learner's  motivation,  and 
thus  their  mental  effort,  involves  increasing  the  personal  relevance  of  the  material.  When  instruction  is  perceived  as 
relevant,  learners  perceive  that  important  personal  needs  are  being  met  by  the  learning  situation"  (p.406,  italics  in 
original) 

Nwagbara  (1993)  presented  learners  with  two  versions  of  a videotape  to  examine  the  effects  of  relevance  on 
’ learners  motivation  toward  learning  the  steps  involved  in  creating  a document  using  a desktop  , ublishing  system.  Half 
the  learners  received  the  "standard"  videotape  and  half  received  a "motivationally  enhanced"  version  of  the  program  that 
included  scenes  to  increase  the  relevance  of  the  lesson  for  the  learners.  He  found  that  learners  reported  a greater 
willingness  to  expend  effort  in  watching  the  motivationally  enhanced  videotape  than  in  watching  the  same  program 
without  the  motivational  enhancements.  Although  the  actual  effort  expended  in  the  lesson  was  not  assessed,  these  results 
suggest  that  increasing  the  perceived  relevance  of  a lesson  may  encourage  learners  to  invest  more  mental  effort  in  learning 
the  material. 

The  purpose  of  this  study  was  to  manipulate  the  degree  of  "relevance"  present  in  both  a videotaped  and  print 
versions  of  a lesson  to  determine  whether  increasing  the  relevance  of  a lesson  increases  the  amount  of  mental  effort 
invested  iri  the  lesson.  In  addition,  mental  effort  was  assessed  using  both  the  secondary  task  technique  and  a self-report 
questionnaire  to  determine  the  more  valid  way  of  assessing  mental  effort.  And  finally,  the  study  was  conducted  to 
determine  the  extent  to  which  Salmon's  conclusions  apply  to  learners  of  college  age.  The  following  research  questions 
were  addressed: 

a) Do  learners  invest  more  effort  in  print-based  versions  of  the  lessons  than  in  video-based  lessons? 

b)  Do  learners  invest  more  mental  effort  in  motivationally  enhanced  lessons  than  in  "standard"  lessons  which 
lack  these  characteristics? 

c)  Do  learners'  self-reports  of  effort  correlate  with  their  "on-line"  measurements  of  effort  collected  using  a 
secondary  task  technique? 

d)  Do  learners  mental  effort  scores  (from  secondary  task  or  self-report  measures)  correlate  with  their  post-test 
scores  on  recall  measures 


Methods 


For  this  study,  a 2 (text  vs.  video)  X 2 (enhanced  vs.  standard  versions)  X 2 (secondary  task  vs.  non-secondary 
task)  design  was  used.  Eight  sessions  were  scheduled  over  a two  week  period  with  treatments  randomly  assigned  to  each 
of  the  eight  sessions. 
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Participants 

Students  enrolled  in  a sophomore  level  education  course  at  a large  Midwestern  University  were  invited  to 
participate  in  a two  hour  lesson  on  desktop  publishing.  Of  225  students  invited  to  participate,  130  students  volunteered 
for  the  study.  Thirteen  cases  were  eliminated  due  to  incomplete  data  (failed  to  complete  the  mental  effort  questionnaire, 
failure  to  respond  to  the  secondary  task,  or  to  problems  with  the  data  collection  equipment).  After  eliminating 
incomplete  data  sets,  117  participants  remained  in  the  study.  Participants  were  both  male  (N=  35)  and  female  (N=82). 

Instructional  Materials: 

Instructional  materials  consisted  of  four  lessons:  a)  a "standard"  video  on  desktop  publishing,  b)  a 
motivationally  enhanced  videotape  , c)  a text  version  of  the  "standard"  lesson,  and  d)  a text  version  of  the  motivationally 
enhanced  lesson. 

The  two  videotapes  were  used  in  Nwagbara’s  (1993)  previous  study.  The  "standard"  video  tape  is  a 
commercially  available  product.  The  motivationally  enhanced  videotape  was  created  by  editing  segments  designed  to 
increase  the  personal  relevance  of  the  material  into  the  commercial  videotape. 

Keller  (1987)  suggests  that  to  increase  the  relevance  of  a lesson,  instructional  designers  should  provide  goal 
orientation,  motive  matching,  and  familiarity.  Goal  orientation  can  be  created  by  relating  the  benefits  of  instruction  to 
getting  a job,  getting  a promotion,  or  improved  job  performance,  or  other  goals  that  may  be  of  value  to  the  learners. 

The  goals  of  the  lesson  should  be  clearly  presented  or  learners  should  have  the  opportunity  to  set  their  own  goals.  In 
addition,  learners  must  understand  how  the  concepts  and  skills  are  related  to  their  present  or  future  goals.  For  learners 
who  are  present  oriented,  the  instructional  content  can  be  related  to  their  current  interests.  For  future  oriented  learners, 
the  instruction  must  provide  a rationale  as  to  how  the  information  will  be  helpful  in  the  future.  Motive  matching  can  be 
enhanced  by  providing  personal  achievement  opportunities,  cooperative  activities,  and  positive  role  models.  Learners 
should  be  encouraged  to  assume  responsibility  for  his  or  her  own  behavior  and  the  materials  should  promote  perceptions 
of  self-improvement  on  the  part  of  the  learners  (McClelland,  1965;  Keller,  1983).  Familiarity  can  be  increased  by  using 
concrete  examples  from  settings  familiar  to  the  learner;  stories,  pictures,  or  testimonials  about  specific  people  or  things 
familiar  to  the  learner;  and  through  the  use  of  analogies  (Keller,  1984). 

Following  Keller's  (1983;  1987)  recommendations,  motivational  elements  were  added  to  a)  illustrate  concrete 
and  provide  practical  examples,  b)  provide  goals  for  the  learners,  c)  provide  positive  role  models  through  testimonials 
from  product  users,  d)  illustrate  how  the  lesson  might  help  the  learner  improve  their  present  and  future  skills,  e) 
encourage  learners  to  assume  responsibility  for  their  learning,  and  e)  illustrate  how  the  skills  are  useful.  For  example,  a 
section  was  added  that  stated: 

" Some  of  you  may  have  used  the  computer  at  home  or  in  school  to  do  some  assignments,  write  term 
papers,  and  other  interesting  things.  You'll  see  that  some  of  the  things  you  already  know  about  computers,  like 
how  to  turn  the  power  on,  how  to  use  the  mouse,  or  your  familiarity  with  the  buttons  on  the  key  board  will  be 
handy  and  helpful  to  you  in  the  process."  (Familiarity) 

"Gaining  the  knowledge  and  skills  in  this  presentation  will  open  a variety  of  windows  of  opportunities  for 
you  in  the  world  of  work  in  terms  of  good  jobs,  good  salaries,  and  of  course,  prospects  for  promotions  whether 
you  chose  to  work  in  the  industry  or  as  the  owner  of  your  won  desktop  publishing  business."  (goal  orientation) 

In  addition,  Nwagbara  a)  removed  non-instructional  graphic  images,  b)  condensed  time  between  the  narrators  and 
c)  eliminated  or  reduced  the  time  between  illustrations  of  the  computer  screens  to  result  in  two  versions  of  the  lesson  that 
were  both  approximately  35  minutes  in  length. 

For  this  study,  text  versions  were  prepared  from  Nwagbara's  tapes.  The  text  of  the  print  versions  was  prepared 
by  transcribing  the  narration  from  the  videotapes.  Two  undergraduate  students  viewed  the  videotapes  and  read  the  text 
versions  to  identify  visual  images  that  they  believed  to  be  important  to  the  text.  Based  on  their  recommendations,  35 
images  were  digitized  from  the  videotape  and  added  to  the  text  to  provide  interest  and  illustrate  key  points.  For  example, 
the  text  booklet  begins  with  a series  of  images  created  using  desktop  publishing  packages.  Visual  images  of  the 
"speaker"  accompany  the  testimonials.  And  finally,  the  text  versions  include  numerous  images  of  the  computer  screen. 
The  verbatim  transcription  was  modified  slightly  for  the  print  version.  When  the  narrator  makes  statements  such  as 
"now  you  see"  , the  text  version  makes  reference  to  the  appropriate  figure  (in  Figure  1,  you  see.."). 
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Text  versions  were  analyzed  to  ensure  that  they  were  equivalent  to  the  videotaped  versions.  One  by  one,  four 
graduate  students  in  instructional  design  independently  reviewed  the  text  and  video  materials.  Each  student  reviewed  the 
regular  text  and  video  versions  together,  then  the  motivationally  enhanced  materials  were  reviewed  together.  As 
differences  were  identified  between  text  and  video  versions,  revisions  were  made  to  the  text  version.  After  two  rounds  of 
revisions,  two  reviewers  agreed  that  the  text  and  video  versions  were  equivalent. 

Assessment  measures 

Self-report  questionnaire:  The  self-report  measure  of  effort  was  created  by  modifying  questions  used  by  other 
researchers  (Salomon,  1983b;  Salomon  & Leigh;  1984;  Sherman,  1993).  When  Salomon's  questions  were  used  in  a 
previous  study  with  college  students,  the  instrument  only  had  Chronbach's  alpha  of  .55  (Cennamo,  Savenye,  & Smith, 
1991).  Beentjas  and  van  der  Voort  (1993)  reported  alphas  of  .60  to  .64  when  using  similar  questions  with  children. 

Thus,  the  instrument  was  modified  in  an  attempt  to  increase  the  reliability  of  the  instrument.  Four  of  the  questions  were 
modifications  of  questions  used  by  Salomon  (1983b,  1984)  and  five  questions  were  modifications  of  questions  used  by 
Sherman  (1993)  who  reported  a Chronbach’s  alpha  of  .69  for  his  mental  effort  questionnaire.  Participants  were  asked  to 
respond  to  questions  such  as  "While  reading  the  booklet,  I concentrated  hard  on  what  was  shown  in  the  pictures."  "The 
lesson  made  me  think  very  hard."  "I  tried  hard  to  understand  the  information  presented  in  this  lesson."  Participants 
responded  on  a 5-point  Likert  scale  which  rated  responses  from  "Strongly  Agree"  (5)  to  "Strongly  Disagree"  (1).  In  this 
study,  the  questionnaire  had  a Chronbach's  alpha  of  .78  , indicating  an  acceptable  degree  of  internal  consistency  (Borg 
and  Gall,  1983). 

Additional  questions  on  the  self-report  questionnaire  gathered  information  regarding  participants'  gender,  previous 
experiencfe  with  Macintosh  computers,  experience  using  desktop  publishing  software,  and  other  demographic  information. 

Secondary  task  measure.  The  secondary  task  measure  consisted  of  a computer  program  that  presented  an  audible 
beep  at  random  intervals  then  measured  and  the  time  between  the  beep  and  the  moment  when  the  participant  clicked  the 
mouse  button  in  "tics"  (a  tic  is  equivalent  to  1/60  of  a second).  During  the  practice  phase,  participants  received  seven 
practice  beeps  randomly  generated  in  10  to  30  second  intervals.  By  averaging  the  seven  practice  response  times,  a 
baseline  response  rate  was  established. 

During  the  actual  use  with  instructional  materials,  participants  responded  to  a beep  randomly  generated  over 
intervals  ranging  from  30  to  90  seconds.  Secondary  task  reaction  times  were  calculated  for  each  student  by  averaging  the 
response  time  for  each  participant  engaged  with  a lesson.  Consistent  with  a procedure  used  by  another  researcher 
(Grimes,  1990),  responses  that  were  more  than  three  standard  deviations  from  the  individual's  mean  were  ignored  for 
analysis.  For  example,  if  a participant  failed  to  respond  to  a beep  or  responded  after  a much  longer  interval  than  usual, 
that  particular  score  was  eliminated  from  the  final  average  score. 

Post-test.  The  post-test  consisted  of  a cued  recall  test,  where  students  wrote  down  all  they  could  remember 
about  topics  presented  in  the  lessons.  Participants  were  first  asked  to  recall  the  ideas  that  impressed  them  most.  The 
next  five  questions  corresponded  to  the  five  major  topics  of  the  videotape.  Participants  responded  to  questions  such  as:  " 
In  SuperPaint,  what  tools  do  you  remember  and  what  were  their  uses?"  and  "What  steps  would  you  follow  to  create  a 
news  letter  with  a page-layout  program?"  Finally  students  were  provided  with  the  opportunity  to  list  other  information 
they  recalled  from  the  lesson  ("What  else  do  you  remember  from  the  lesson?"  ) 

A scoring  key  was  developed  that  categorized  responses  to  the  post-test.  The  primary  researcher  and  two  research 
assistants  (a  graduate  student  in  instructional  design  and  an  undergraduate  education  student)  each  reviewed  15  recall  post- 
tests and  clustered  responses  that  seemed  to  be  of  a similar  type.  Responses  seemed  to  directly  recall  information 
presented  in  the  lessons,  make  inferences  based  on  the  information  presented  in  the  lessons,  summarize  information  that 
was  presented  in  the  lessons,  or  evaluate  the  lessons.  Differences  in  coding  responses  among  the  three  individuals  were 
discussed  and  coding  categories  were  finalized.  The  final  coding  categories  consisted  of  recall  responses  (this  software 
leaves  room  to  correct  mistakes,  I can  design  my  own  letterhead),  inferences  (it  seemed  easy,  it  seems  convenient  to  use), 
and  summary  statements  (there  were  different  photos  of  parts  of  the  computer,  I remember  the  letters  from  people  in  the 
industry).  Evaluative  comments  (the  information  was  presented  in  good  form;  I was  falling  asleep)  were  not  coded.  The 
two  research  assistants  then  coded  15  other  post-test  using  the  coding  scheme.  After  finding  an  acceptable  inter-rater 
reliability  of  .97  between  the  two  scorers,  the  tests  were  scored  by  one  of  the  research  assistants  using  the  scoring  key. 
Participants  received  a recall  score,  inference  score,  summary  score,  and  total  post-test  score. 
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Procedures 

All  sessions  were  held  in  a computer  lab  which  contained  approximately  40  computers.  Groups  using  the 
secondary  task  measure  had  a demonstration  of  the  data  collection  program  and  practice  with  the  program.  During  the 
practice  session,  a baseline  measure  of  reaction  time  to  the  secondary  task  was  collected. 

A]]  participants  received  either  a motivationally  enhanced  videotape,  standard  videotape,  motivationally 
enhanced  print  booklet,  or  "standard"  print  booklet.  All  learners  were  asked  to  read  or  view  the  lesson  to  learn  as  much  as 
possible  about  the  topic  of  the  lesson.  Participants  in  four  of  the  eight  sessions  were  asked  to  respond  to  a secondary 
task  while  attending  to  their  lesson.  They  were  informed  that  their  primary  task  was  to  learn  as  much  as  possible  from 
the  lesson  and  also  to  press  a key  on  a computer  keyboard  as  soon  as  they  heard  a beep.  Participants  participating  in  the 
secondary  task  sessions  wore  earplugs  to  avoid  sound  carrying  throughout  the  room. 

Participants  receiving  one  of  the  videotaped  versions  viewed  the  lesson  as  a group  from  a large  video  monitor 
placed  in  the  front  of  the  room.  Participants  receiving  the  text  versions  read  the  lesson  at  their  own  pace  from  individual 
booklets.  Following  completion  of  one  the  four  versions  of  the  lesson,  all  participants  completed  the  mental  effort 
questionnaire.  Finally,  all  participants  completed  the  post-test,  then  created  a logo  and  letterhead  using  the  software 
demonstrated  in  the  lessons.  The  participants  were  provided  an  opportunity  to  apply  the  skills  demonstrated  in  the  lesson 

to  create  a logo  and  letterhead;  however,  these  products  were  not  analyzed. 

Results 

Multivariate  Analysis  of  Variance  (MANOVA)  was  used  to  determine  if  there  were  significant  differences  among 
the  eight  groups  for  post-test  and  self-report  mental  effort  scores.  The  analysis  indicated  several  significant  effects. 

Interaction  hetwren  media  and  version  for  post-test  scores.  The  interaction  between  media  (text  or  video)  and 
lesson  version  (standard  or  motivationally  enhanced  ) was  significant,  F (2,  107)  = 4.37,  p=.01.  Univariate  analysis 
indicated  that  the  interaction  effects  were  caused  by  significant  differences  among  groups  on  the  post-test  scores,  F (1, 
108)=  8.38,  p=.005.  An  examination  of  the  means  (see  Table  1 ) revealed  that  post-test  scores  were  higher  for  the 
standard  version  of  video  (M=  30.58)  than  the  motivationally  enhanced  version  (M=  21.23) , and  higher  for  the 
motivationally  enhanced  version  of  text  (M=29.64)  than  for  the  standard  version  (M=  24.49).  (See  Figure  1 .) 

Figure  1;  Mean  Post-test  Scores 
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Mental  effort  assessment  condition  and  recall  scores.  There  were  also  significant  differences  in  the  post-test 
scores  between  groups  who  responded  to  the  secondary  task  measure  of  effort  and  those  who  only  responded  to  the  mental 
effort  questionnaire  , F (2,  1 07)=  4. 1 0,  p=.02.  Univariate  analysis  revealed  that  the  significant  difference  was  due  to 
significant  differences  in  achievement  test  scores,  F (1,108)  = 6.71,  p=.01 . Participants  who  responded  to  the  secondary 
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task  while  reading  or  viewing  the  lesson  received  significantly  lower  scores  (M=  23.33)  on  the  post-test  of  cued  recall 
than  those  participants  who  only  responded  to  the  mental  effort  questionnaire  after  the  lesson  (M=  28.70).  (See  Table  1.) 

Table  1:  Post-test  means 


standard  enhanced 

text text 


Secondary 

23.94 

27.67 

task 

10.80 

9.53 

17 

12 

Questionnaire 

25.00 

30.76 

13.24 

12.17 

18 

21 

Totals 

24.49 

29.64 

11.95 

11.23 

35 

33 

standard 

video 

enhanced 

video 

TOTAL 

23.11 

15.75 

23.33 

8.77 

11.60 

10.66 

9 

8 

46 

34.53 

24.36 

28.70 

12.98 

13.36 

13.29 

17 

14 

31 

30.58 

21.23 

12.77 

13.16 

26 

22 

Effect  of  media  on  mental  effort  scores.  MANOVA  indicated  a significant  effect  for  media,  F(2,  107)=  6.83, 
p=.002.  Univariate  analysis  revealed  that  the  significant  effect  was  due  to  differences  among  groups  on  their  responses  to 
the  mental  effort  questionnaire,  F(  1,  108)=  1 1.78,  p=.001.  Students  who  received  the  text  version  of  the  lesson  reported 
spending  significantly  more  effort  (M=29.53)  in  the  lesson  than  those  who  received  the  videotaped  version  (M=26.23). 
(See  Table  2.) 

Table  2:  Means  on  Mental  Effort  Questionnaire 

Texl Video 

Means  29.53  26.23 

SD  5.59  4.52 

N 68  48 


Note:  Possible  scores  ranged  from  9 to  45 

Secondary  task  data.  Initially,  Analysis  of  Variance  was  used  to  analyze  differences  among  group  means  for 
baseline  reaction  times  collected  during  the  practice  session.  There  were  no  significant  differences  in  the  baseline  reaction 
times  among  groups  for  media,  F(  1,  42)  = 1.43.  p=.24,  lesson  version  F(l,  42)=  .08.  p=.78,  or  the  interaction  between 
media  and  lesson  version,  F(l,  42)=  .31,  p=.58.  Since  there  were  no  significant  differences  among  groups  for  the 
baseline  reaction  times  (see  Table  3 for  mean  scores  on  practice  task),  Analysis  of  Variance  was  used  to  determine 
differences  among  group  means  for  reaction  times  to  the  secondary  task.  The  interaction  between  media  and  lesson 
version  was  significant,  F(l,42)=  4.17,  p=.047.  There  also  were  significant  differences  among  groups  for  media,  F 
(1,42)=  7.28,  p=.01,  and  lesson  version,  F(l,42)=  1 1.39,  p =.002.  An  examination  of  the  group  means  revealed  that 
reaction  times  for  those  who  received  the  video  versions  were  significantly  longer  (M=  78.13)  than  those  who  received 
the  text  versions  (M=55.38),  In  addition,  the  reaction  time  means  for  those  who  received  the  motivationally  enhanced 
versions  of  the  lesson  (M=78.33)  were  significantly  longer  than  those  who  received  the  standard  versions  of  the  lesson 
(M=52.59).  (See  Table  4 for  means  for  secondary  task.)  Although  the  analysis  revealed  a significant  interaction  between 
media  and  lesson  version  (see  Figure  2),  an  examination  of  the  means  indicates  that  the  effects  of  media  and  lesson 
version  was  much  stronger  than  the  interaction.  The  significant  interaction  effect  appears  to  be  caused  by  the  larger 
difference  in  reaction  time  scores  between  the  standard  video  group  (M=56.27  ) and  the  enhanced  video  group  (M=  102.7 1) 
than  between  the  standard  text  group  (M=  50.65)  and  the  enhanced  text  group  (M=  62.08). 
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Table  3;  Mean  baseline  reaction  scores 


Tahiti 


O 

ERIC 


Text Video 


Standard 

43.62 

47.76 

SD 

14.24 

18.05 

n 

17 

9 

Enhanced 

41.84 

53.24 

SD 

27.50 

25.45 

n 

12 

8 

Note:  reaction  time  measured  in  ’’ties'’,  equivalent  to  1/60  of  a second 


4:  Mean  reaction  times  to  secondary  task 

Text Video Total 


Standard 

50.65 

56.27 

52.59 

sd 

18.68 

23.83 

20.31 

n= 

17 

9 

26 

Enhanced 

62.08 

102.71  78.33 

sd 

34.42 

36.79 

40.02 

n= 

12 

8 

20 

Totals 

55.38 

78.13 

26.41 

38.04 

29 

17 

Note:  reaction  time  measured  in  ’’tics”,  equivalent  to  1/60  of  a second 
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Figure  2:  Mean  Reaction  Times  to  Secondary  Task 


Correlations.  Correlations  among  scores  on  the  three  dependent  measures  (post-test,  mental  effort  questionnaire, 
and  secondary  task  reaction  measure)  were  tested  using  a Pearson's  r test  of  correlation.  The  correlation  between  reaction 
time  means  on  the  secondary  task  measure  and  the  scores  on  the  self-report  of  mental  effort  were  very  low,  r=-.05,  p=.61 
(N=46).  Likewise,  the  correlations  were  low  between  post-test  scores  and  responses  to  the  self-report  measure  of  effort 
r=-.069,  p=  .46  (n=l  17).  The  correlation  between  post-test  scores  and  the  secondary  task  response  times  was  somewhat 
higher,  r=-.28,  p=  .056  (n=  46)  but  in  a negative  direction.  Correlations  between  the  number  of  inferences  generated  on 
the  post-test  and  scores  on  the  mental  effort  questionnaire  and  secondary  task  measure  were  also  low,  with  values  of  r=- 
.06,  p=.79  (questionnaire)  and  r=  .22,  p=  .52  (secondary  task)  respectively. 

Post  hoc  analysis 

Several  analysis  were  conducted  to  determine  whether  participants'  prior  experience  with  desktop  publishing  or 
Macintosh  computers  may  have  influenced  participants  scores  on  the  post-test  and  mental  effort  questionnaire.  When 
prior  experience  with  desktop  publishing  (experienced/  no  experience)  and  Macintosh  computers  (experienced/  no 
experience)  were  used  as  independent  variables  in  a MANOVA  with  mental  effort  and  post-test  scores  as  the  dependent 
variables,  there  were  no  significant  interactions,  F(2,  103)=  1.20,  p=.31,  differences  due  to  prior  experience  with 
Macintosh  computers,  F(2,  103)=  2.17,  p=.12,  or  desktop  publishing  software  (F(2,103)  = .32,  p=.73). 

Discussion 

The  purpose  of  this  study  was  to  investigate  a)  whether  college-age  learners  invest  more  effort  in  print-based 
lessons  designed  to  teach  the  skills  involved  in  desktop  publishing  than  in  video-based  versions  of  the  lessons,  b)whether 
learners  invest  more  mental  effort  in  motivationally  enhanced  lessons  than  in  "standard"  lessons  which  lack  motivational 
enhancements,  c)  the  extent  to  which  learners'  self-reports  correlate  with  their  "on  line"  measurements  of  effort  collected 
using  a si  condary  task  technique  and  d)  the  correlation  between  mental  effort  scores  (from  secondary  task  or  self-report 
measures)  and  post-test  scores  on  a recall  measure. 

As  in  Beentjas  and  van  der  Voort's  (1993)  study,  the  two  measures  of  mental  effort  appeared  to  assess  different 
things.  Correlations  between  the  mental  effort  questionnaire  and  the  secondary  task  measure  of  effort  were  very  low. 
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Despite  the  lack  of  correlation  between  scores  on  the  mental  effort  questionnaire  and  the  secondary  task  measure 
of  effort,  the  findings  of  this  study,  conducted  with  college  age  students,  are  consistent  with  the  results  of  studies 
conducted  with  elementary  and  middle  school  students.  Like  the  younger  learners  who  participated  in  Salpmon  (1983; 
1984;  Salomon  & Leigh,  1984)  and  Beentjas  and  van  der  Voort's  (1993)  studies,  college  students  who  participated  in  this 
study  reported  that  they  invested  more  mental  effort  in  learning  from  print  versions  of  the  lessons  than  in  video-based 
versions  on  a self-report  questionnaire.  And  like  the  younger  learners  who  responded  to  a secondary  task  measure  of 
effort  in  Beentjas  and  van  der  Voort's  (1993)  study,  the  college  students  who  received  the  video-based  lesson  had 
significantly  longer  reaction  times  to  the  secondary  task  measure. 

But  the  question  remains:  Which  is  a more  valid  way  of  assessing  mental  effort?  If  sensitivity  to 
manipulations  designed  to  increase  the  effort  expended  in  the  lesson  is  used  as  an  indictor  of  the  most  valid  measure  of 
mental  effort,  then  the  secondary  task  measure  may  be  superior.  Using  this  means  of  assessing  mental  effort, 
participants  spent  more  mental  effort  in  reading  and  viewing  the  motivationally  enhanced  versions  of  the  lesson  than  in 
the  standard  versions. 

However,  the  results  of  this  study  indicate  that  the  secondary  task  measure  of  effort  interferes  with  student 
learning.  For  students  who  responded  to  the  secondary  task  while  reading  or  viewing  the  lesson,  overall  achievement  was 
lower  than  for  the  other  groups.  Constant  response  to  the  secondary  task  may  have  interrupted  their  concentration  just 
enough  to  disrupt  the  learning  process. 

In  addition,  there  appears  to  be  very  little  relationship  between  learner's  achievement  test  scores,  as  measured  by 
a test  of  cued  recall,  and  the  extent  to  which  learners  are  cognitively  engaged  in  the  lesson,  as  measured  by  a secondary 
task  technique.  Likewise,  there  appeared  to  be  very  little  relationship  between  learners'  achievement  scores  and  their 
perceptions  of  the  mental  effort  they  expended  in  processing  the  lesson  as  reported  on  a self-report  questionnaire. 

However,  these  results  are  consistent  with  the  low  correlations  between  achievement  scores  and  mental  effort  found  in 
Beentjas  and  van  der  Voort's  (1993)  research  with  younger  learners. 

Of  course  recall  depends  not  only  on  the  extent  to  which  learners  concentrate  on,  process,  or  elaborate  on  the 
materials  but  also  on  their  ability  and  willingness  to  retrieve  the  information  (Driscoll,  1994).  Participants  may  have 
lacked  motivation  to  succeed  on  the  achievement  test,  as  course  credit  was  awarded  for  participation  in  the  study  regardless 
of  scores  on  the  achievement  test. 

Keller  (1983)  also  reminds  us  that  achievement  scores  reflect  a variety  of  factors  other  than  the  extent  to  which 
learners'  expend  effort  in  processing  a lesson.  He  (1983)  suggests  that  effort,  rather  than  performance,  is  an  indicator  of 
motivation.  Although  participants  who  received  the  motivationally  enhanced  videotape  expended  more  effort  in  the 
lesson,  as  measured  by  a secondary  task  technique,  than  those  who  received  any  other  version,  recall  scores  were  lowest 
for  those  who  received  the  motivationally  enhanced  videotape. 

The  overall  intent  in  adding  "motivational  enhancements"  was  to  increase  the  personal  relevance  of  the  materials 
(Nwagbara  1993),  theoretically  increasing  the  mental  effort  expended  on  the  materials  and  achievement  scores.  However, 
the  addition  of  motivational  aspects  may  have  "backfired".  With  the  addition  of  motivational  enhancements,  learners  may 
have  had  a difficult  time  distinguishing  relevant  information  from  irrelevant  information.  For  example,  learners  who 
received  the  motivationally  enhanced  videotape  may  have  been  thinking  about  how  desktop  publishing  could  enhance 
their  lives  and  missed  information  necessary  for  success  on  the  recall  test.  Perhaps  increased  cognitive  engagement, 
especially  in  response  to  video,  resulted  in  lower  achievement  scores.  As  learners  took  the  time  to  elaborate  on  the 
information  presented  in  the  motivationally  enhanced  videotape,  they  ran  the  risk  of  missing  critical  information  due  to 
the  constant  stream  of  visual  and  auditory  information  presented  via  video.  Although  learners  who  received  the  standard 
version  of  the  videotaped  lesson  recalled  more  information  than  those  who  received  the  motivationally  enhanced  video 
version,  learners  who  received  the  motivationally  enhanced  version  in  print  form  recalled  more  information  than  those 
who  received  the  standard  version  of  the  print  lesson.  In  addition,  learners  who  received  the  motivationally  enhanced  print 
version  had  longer  reaction  times  to  the  secondary  task,  suggesting  greater  mental  effort,  than  those  who  received  the 
standard  print  version.  When  learning  from  print,  readers  may  take  time  to  elaborate  on  the  content  of  a lesson  without 
"missing"  critical  information,  due  to  the  individual's  control  over  the  pace  of  their  reading.  This  study  provided  no 
information  on  the  nature  of  thoughts  that  may  have  occurred  during  the  lesson.  Future  research  using  a "think-aloud" 
protocol  may  provide  additional  insight  as  to  the  cognitive  processing  that  occurs  while  reading  or  viewing  a lessons. 

At  minimum,  there  is  a complex  relationship  between  effort  and  achievement.  In  some  studies,  longer  reaction 
times  to  a secondary  task  have  paralleled  less  achievement  (Lang,  et.  al.,  1993;  Reeves,  et.  al„  1985,  and  others); 
however,  in  other  situations,  increased  effort  paralleled  increased  achievement  (Grimes,  1990;  Britton,  Glynn,  Muth,  & 
Penland,  1985,  and  others).  Other  times,  more  or  less  effort  made  no  difference  in  achievement  (Britton,  Glynn,  Meyer, 


& Penland,  1982;  Reeves,  et.  al.,  1985).  Does  the  secondary  task  technique  and  achievement  tests  the  simply  measure 
different  things?  The  low  correlations  between  learners'  post-test  scores  and  their  reaction  times  to  the  secondary  task 
appear  to  support  this  idea. 

Future  researchers  may  need  to  decide  which  is  important  to  them:  Effort  or  achievement?  Does  effort  matter 
without  achievement?  Does  achievement  matter  if  learners  are  cognitively  engaged  and  expending  effort  in  learning?  It 
may  not  be  useful  to  use  the  secondary  task  technique  to  determine  if  manipulations  increase  the  amount  of  effort 
invested  in  learning  from  materials  in  situations  where  the  major  goal  is  achievement. 

Additional  problems  in  using  the  secondary  task  measure  of  effort  remain.  Large  standard  deviations  in  scores 
are  common  when  using  a secondary  task  measure  of  effort.  Due  to  the  small  number  of  students  in  this  study,  and  the 
large  standard  deviations,  the  results  of  this  study  should  be  replicated  with  a larger  number  of  students  to  increase 
confidence  in  the  findings.  In  this  study,  it  was  possible  to  nold  the  class  in  a computer  lab  to  avoid  the  appearance  of  a 
laboratory  setting  because  the  subject  matter  of  the  lessons  was  computer  skills;  however,  researchers  must  consider  that 
the  use  of  the  secondary  task  measure  requires  a laboratory  like  setting,  where  participants  have  access  to  the  data 
collection  equipment.  In  addition,  there  is  a great  deal  of  variability  in  the  way  researchers  score  and  analyze  data  from 
the  secondary  task.  If  investigations  of  ways  to  increase  the  effort  invested  in  video-based  materials  are  to  continue,  the 
search  for  techniques  to  accurately  assess  the  effort  expended  in  the  lessons  continues  to  be  an  important  area  for  further 
study. 
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Some  of  the  elements  instructional  designers  must  consider  when  designing  screens  for  computer  based  instruction 
(CBI)  include  legibility,  readability,  and  particularly  font  size.  These  elements  are  also  factors  that  influence  the 
comprehension  and  speed  with  which  people  learn  (Grabinger,  1992).  Although  Grabinger  proposes  that  no  single 
clement  promotes  learning  more  than  another;  the  combination  of  these  elements  on  the  computer  screen  affects  learning 
more  than  any  single  element.  The  contribution  of  font  size  in  the  development  of  CBI  deserves  investigation.  The 
purpose  of  this  study  was  to  determine  which  size  font  is  most  effective  for  computer  based  instruction  users. 

Several  research  studies  were  conducted  between  1885  and  1963  on  the  legibility  of  print  based  information.  Tinker 
(1963)  reported  that  no  single  method  of  measurement  is  adequate  for  determining  the  legibility  of  print  in  all  kinds  of 
typographical  setups.  Tinker  provides  several  guidelines,  based  on  the  research  prior  to  1963,  for  the  optimal  legibility  in 
print  materials.  These  guidelines  include  (a)  leading  (space  between  lines)  has  an  important  effect  on  the  legibility  of 
tvpe,  especially  in  fonts  smaller  than  12  point  and  (b)  font  sizes  between  8 and  12  points  are  acceptable.  Eleven  point 
font  is  the  best  size  for  reading  speed  and  comprehension. 

Many  of  the  recommendations  offered  in  the  literature  lack  research  support.  One  example  is  the  recommendation  of 
Felker,  Pickering,  Charrow,  and  Holland  (1981)  of  8 to  10  point  font  for  the  printed  page  of  documents,  and  14  point 
font  for  headings  on  documents.  These  authors  reported  the  most  effective  font  size  should  be  easy  to  read  and  pleasing 
to  the  eye  but  offered  no  real  evidence  of  what  constituted  ease  of  reading  and  what  is  pleasant  to  the  eye. 

A review  of  literature  revealed  very  little  direct  research  on  computer  font  size.  Similarly  there  was  only  limited 
research  concerning  font  size  for  television  screens.  A television  screen  is  an  interface  similar  to  a computer  monitor  but 
there  are  some  major  differences.  These  differences  are  television  is  not  typically  used  for  extensive  reading.,  the  viewer  to 
screen  distance  is  usually  greater  than  the  typical  distance  of  one  to  three  feet  for  a computer  screen,  and  the  resolution 
and  the  physics  of  screen  refresh  for  a television  screen  is  different  from  a computer  screen. 

Although  the  television  research  results  do  not  apply  directly  to  this  study,  some  of  the  techniques  and  concepts 
from  this  research  do  apply.  One  of  these  is  the  more  general  "subtended  angle”  approach  rather  than  trying  to  identify 
specific  fonts  for  specific  distances.  Nevertheless,  this  study  had  a different  medium  and  different  tasks  to  study 

With  the  advent  of  television,  researchers  began  studying  the  legibility  of  characters  on  television  screens.  Smith 
(1979)  showed  a proportional  width  font  is  significantly  better  for  recognition  of  words  on  TV  screens  than  a non- 
proportional font.  Research  shows  that  character  size  ranging  from  10  to  24  minutes  of  vertical  arc  is  best  for  legibility 
on  television  screens  (Shurtleff,  1967).  Minutes  of  arc  refer  to  the  angle  defined  by  the  top  of  the  character,  the  eye,  and 
the  bottom  of  the  character.  This  angle  is  the  subtended  angle. 

Shurtleff  (1967)  reported  on  legibility  and  symbol  size  on  television  screen  in  a review  of  literature  that  gave 
guidelines  for  accuracy  of  reading  to  occur.  The  examination  found  that  viewers  required  a visual  size  of  between  1 2 and 
1 5 minutes  of  arc  and  vertical  resolution  between  8 and  12  scanning  lines  per  symbol  height. 

Legibility  involves  perceiving  letters  and  words,  and  the  reading  of  continuous  textual  material.  In  the  final 
analysis,  optimal  legibility  of  print  is  achieved  by  typographical  arrangement  in  which  shape  of  letters  and  other 
symbols,  characteristic  word  forms,  and  all  other  typographical  factors  such  as  type  size,  line  width,  leading,  page  layout, 
and  density  are  coordinated  to  produce  comfortable  vision,  and  easy  rapid  reading  with  comprehension. 

Computer  technology  is  rapidly  becoming  a component  in  training  and  education.  People  are  using  computers  to 
learn  new  concepts  and  reinforce  previously  learned  cognitive  and  motor  skills.  One  such  issue  of  interest  is  the 
legibility  of  the  text  on  the  computer  screen.  Legibility  affects  several  aspects  of  learning,  including  the  reading  speed  of 
the  user  and  the  comprehension  of  the  information  by  the  user  (Tinker,  1963).  Therefore,  the  legibility  of  the  text  on 
the  computer  screen  is  an  item  of  importance  to  designers  of  computer  based  instruction. 

Several  authors  (Heines,  1984;  Hooper  & Hannafin,  1986;  Criswell,  1989)  provide  guidelines  for  optimal  font 
size,  leading,  blank  space,  and  line  justification  on  computer  screens.  These  guidelines  are  similar  across  the  literature  and 
are  as  follows:  font  size  for  computer  screens  should  be  10  or  12  point,  leading  should  be  15%  of  the  character  width, 
and  left  justification  should  be  used,  leaving  the  right  edge  of  the  screen  jagged  (Brockman,  1990).  There  is  some 
difference  in  opinion  about  the  recommended  length  of  the  line,  with  Hooper  and  Hannafin  (1986)  recommending  up  to 
80  characters  per  line  (the  entire  screen  width)  and  Brockman  (1990)  and  Merrill  (1982)  recommending  that  white  space  be 
left  on  the  edges  of  the  screen.  While  these  guidelines  are  useful  for  designers  of  computer  based  instruction,  there  is 
little  research  using  computer  screens  to  support  these  guidelines.  This  study  investigated  the  relationship  of  font  size, 
comprehension,  and  user  preference. 

One  aspect  of  legibility  is  the  font  size  of  the  text  displayed.  While  there  is  research  relating  to  font  size  for  printed 
information,  little  is  available  on  font  size  as  related  to  legibility  when  text  is  displayed  on  the  computer  screen.  There 
are  guidelines  provided  for  use  of  font  size  on  computer  screens,  but  these  guidelines  tend  to  be  based  on  print  research 
(Hooper  & Hannafin,  1986). 


Giddings  (1972)  conducted  a legibility  experiment  in  an  effort  to  determine  optimum  height  for  characters  under 
particular  conditions.  Giddings'  experiment  attempted  to  find  suitable  sizes  for  words  and  numbers  appearing  on  a 
directly  viewed  CRT.  The  results  indicated  that  differences  were  found  in  the  legibility  of  6 letter  words  and  single  digits. 
Giddings  reported  that  this  may  be  due  more  to  the  informational  content  of  the  words  and  digits  than  their  fundamental 
nature. 

Snyder  & Taylor  (1979)  conducted  similar  research  concerning:  font  size  and  dot  luminance  of  alphanumeric 
characters  in  comparison  to  recognition  accuracy,  response  time,  tachistoscopic  recognition,  and  threshold  visibility  when 
presented  in  a non-contextual  form  on  a CRT  display.  Tachistoscopic  recognition  refers  to  the  length  of  time  a display 
remains  on  a viewer  screen.  Threshold  visibility  pertains  to  the  distance  at  which  a viewer  can  recognize  characters  on  a 
viewer  screen.  Findings  indicated  accuracy  improved  as  font  size  increased  and  regressed  as  distance  increased.  The  effect 
of  higher  levels  of  luminance  was  greater  accuracy  for  the  smaller  character  sizes  and  the  further  distances  from  the  display 
screen.  Response  time  increased  as  character  size  decreased  and  viewing  distance  increased. 

Further,  increases  in  luminance  improved  tachistoscopic  recognition  accuracy  more  for  the  smaller  character  sizes 
than  larger.  No  significant  findings  were  produced  for  threshold  visibility.  The  conclusion  by  Snyder  & Taylor  (1979) 
was  that  characters  less  than  4.79  mm  in  height  at  a viewing  distance  of  1.5  m will  significantly  reduce  legibility. 

Similar  research  by  Beldie,  Pastoor,  & Schwarz  (1983)  examined  the  question:  Are  variable-matrix  characters  more 
legible  than  fixed-matrix  letters  on  readability,  error  identification,  and  the  ability  to  locate  the  position  of  the  target  word 
on  a CRT  screen?  In  this  experiment,  fixed  width  characters  occupied  the  same  amount  of  horizontal  space.  Variable 
width  proportional  characters  occupied  differing  width  spaces  depending  on  the  character.  Variable  width  characters  such 
as  "i"  occupied  less  horizontal  space  than  "w."  The  findings  suggested  that  a proportional  width  font,  even  without  serifs 
is  significantly  better  for  recognition  of  words  on  television  screens.  The  authors  recommended  it  for  text  displays. 

Hathaway's  (1984)  literature  review  revealed  interesting  findings  regarding  letter  size  and  density  of  display  text.  In 
a review  of  Smith’s  (1979)  research,  Hathaway  examined  letter  size  as  it  interfaced  with  legibility.  Smith  found  that  the 
viewers  sitting  24  inches  from  a computer  screen  would  need  letters  no  larger  than  3/16",  and  94  percent  of  viewers 
would  need  letters  no  larger  than  3/32"  high  from  the  same  distance. 

Readability 

Comprehension  and  readability  are  important  factors  when  designing  screen  layout  and  selecting  the  most  effective 
size  of  font.  Presenting  lessons  effectively  on  the  screen  is  one  of  the  central  problems  in  computer  based  learning 
(Isaacs,  1987).  Although  font  size  is  an  important  issue  when  designing  computer  screens,  the  instructional  designer 
must  consider  typographic  factors,  organization  factors,  cueing  factors,  and  control  factors,  in  addition  to  the  content 
(Grabinger,  1989). 

Density  of  displayed  text  on  a CRT  impacts  the  speed  with  which  readers  comprehend  material  (Kolers,  1981; 
Jonassen,  1982;  Hooper  & Hannafin,  1986).  Subjects  read  faster  and  more  accurately  with  double  spaced  text  than  with 
single  spaced  text.  Koler's  findings  indicated  80  characters  per  line  resulted  in  faster  and  more  accurate  reading.  Both  the 
Jonassen,  and  the  Hooper  & Hannafin  studies  suggested  that  ragged  right  margins  are  easier  to  read  than  right  justified 
lines. 

Milheim  & Lavix  (1992)  suggest  using  various  type  sizes  and  fonts  to  emphasize  certain  materials  and  variety. 
These  suggestions  include  using  bold  type  face  for  higher-level  information  such  as  titles  or  headings  and  using  mixed- 
case  letters  for  higher  legibility  and  faster  reading.  Further,  Bailey  and  Milheim  (1991)  recommend  using  only  one  font 
style  per  screen,  text  sizes  larger  than  12  point,  and  primarily  using  New  York  or  Gene.va  fonts.  Faiola  (1990)  suggests 
using  screens  that  are  consistent  in  size,  color,  and  fonts  for  text,  since  more  than  two  or  three  typefaces  and/or  sizes  used 
within  one  screen  can  be  disconcerting  to  learners. 

Grabinger  (1988,  1992)  studied  the  effects  of  single  and  multiple  dimensions  on  the  computer  screen  in  relation  to 
learners'  ability  to  read  a computer  screen  (readability)  and  the  understanding  and  learning  that  followed  the  viewing  of  the 
screen  (study-ability).  The  purpose  of  this  research  was  to  identify  constructs  that  could  guide  the  design  of  computer 
screens  used  to  display  information  in  computer-assisted  instruction,  hypermedia,  or  on-line  help  applications.  Grabinger 
reported  comprehensibility  is  the  combination  of  readability  and  study-ability.  He  further  reported  that  single  element 
research  is  extremely  important  in  identifying  the  strengths,  weaknesses,  and  potential  problems  of  using  specific 
attributes  on  CRT  screens,  such  as  font  size. 

Grabinger's  findings  indicate  that  screens  that  arc  organized  according  to  functional  areas  with  space,  boxes,  and 
lines,  the  use  of  headings,  directive  cues,  and  spaced  paragraphs  combined  to  present  planned,  controlled,  organized,  and 
structured  appearances  are  the  best  for  learning  conceptual  ideas.  Further,  visual  interest  is  an  important  element. 
Grabinger  reported  that  screens  that  are  plain,  simple,  unbalanced,  and  bare  are  perceived  as  undesirable,  while  a moderate 
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degree  of  complexity  creates  an  environment  that  invited  further  exploration,  in  other  words,  motivation  to  continue 
reading. 

On  the  other  hand,  in  related  research,  Morrison,  Ross,  & O’Dell  (1988)  suggest  that  low  density  narratives 
promote  better  learning  and  more  favorable  attitudes  on  CBI  lessons  by  reducing  reading  and  cognitive  processing  demand 
of  screen  displays.  The  notion  here  is  instructional  designers,  at  the  time  of  publishing,  use  the  same  design  formats  and 
teaching  strategies  traditionally  incorporated  in  print  lessons  for  developing  CBI  lessons.  The  rationale  behind  the  study 
is  that  computers  impose  constraints  on  the  display  of  instructional  text  because  they  offer  considerably  less  flexibility 
than  books.  Computers  limit  the  visible  display  to  one  page  at  a time,  make  backward  paging  for  review  more  difficult, 
and  limit  the  size  of  the  page  layout  to  24  lines  and  40-80  characters. 

Pastoor,  Schwarz,  & Beldie  (1983)  conducted  research  on  font  sizes  presented  on  standard  home  color  television 
sets.  Subjects  performed  five  different  tasks  and  rated  the  various  character  sizes.  Their  qualitative  performance  for  all 
tasks  was  the  same  with  each  of  the  different  character  sizes;  the  amount  of  time  necessary  to  complete  the  task,  however, 
varied  up  to  more  than  20%.  This  would  seem  to  indicate  that  none  of  the  character  sizes  exhibited  deficiencies  that 
would  exclude  their  use  on  color  television  screens.  However,  considering  the  time  variance  measured,  and  the  fact  that 
the  larger  font  sizes  decreased  the  time  with  which  subjects  could  perform  the  tasks,  the  larger  fonts  appeared  to  be  better 
for  use  on  television  screens.  In  all  tasks,  smaller  character  sizes  produced  poorer  performance. 

Research  Questions 

The  purpose  of  this  study  was  to  find  a font  size,  or  sizes,  to  recommend  for  presenting  text  on  a computer 
monitor.  There  are  some  obvious  limits  on  the  font  size  because  of  the  discriminatory  limits  of  the  eye  on  the  inability 
of  the  pixel  resolution  of  a monitor  to  differentiate  one  character  from  another  at  extremely  small  sizes.  There  is  also  a 
limitation  at  which  font  sizes  are  too  large  to  read.  In  the  extreme,  a 1000  point  font  would  be  almost  15  inches  tall, 
much  too  tall  to  see  even  one  letter  on  most  computer  monitors.  Somewhere  in  between  these  extremes  are  the 
appropriate  font  sizes  that  users  normally  see  on  computer  screens,  such  as  12  point,  16  point,  and  24  point.  The 
assumption  is  there  is  some  determinable  range  of  these  font  sizes  that  are  readable,  and  a subset  of  the  range  that 
especially  facilitates  reading  speed  and  comprehension.  Finding  the  appropriate  subset  was  the  goal  of  this  research. 

Another  question  of  interest  is  whether  users  might  prefer  one  font  size  over  another  when  reading  for  speed  and 
comprehension.  If  there  are  several  font  sizes  that  promote  optimal  reading  speed  and  comprehension,  user  preference 
might  be  a factor  to  help  direct  a font  choice. 


Methodology 

Participants 

The  sample  consisted  of  one  hundred  and  sixty-three  undergraduate  and  graduate  students  enrolled  in  an  upper-level 
computer  course  entitled  "Classroom  Applications  of  Educational  Technology"  at  a western  university.  The  age  of  the 
volunteer  subjects  ranged  from  20  to  46. 

Subjects  were  randomly  assigned  to  one  of  four  treatment  groups.  Each  treatment  group  received  a different  font 
size.  Group  one  with  47  subjects  used  10  point  font.  Group  two  with  37  subjects  used  12  point  font.  Group  three  with 
47  subjects  used  14  point  font.  Group  four  with  32  subjects  used  16  point  font. 

All  subjects  received  a computerized  version  of  the  comprehensive  section  of  the  Neson-Denny  Reading  Test  Form 
E (1991).  The  comprehensive  section  of  the  Nelson-Denny  Reading  Test  consists  of  eight  passages  with  one  to  eight 
paragraphs  in  each  followed  by  four  to  eight  questions.  The  passages  were  presented  in  a scrolling  window  on  the  screen. 
When  the  student  completed  reading  the  passage,  the  student  moved  to  another  screen  and  answered  the  questions  on  the 
passage.  The  student  could  not  go  back  to  the  passage  and  review.  This  procedure  was  repeated  for  all  eight  passages. 

The  passages  were  presented  on  Apple  Macintosh  Centris  model  610  and  650  computers  with  14  inch  display 
monitors.  The  passages  were  displayed  with  mixed  case  black  letters  on  a white  background  in  a Times  Truetype  font 
style.  The  font  sizes  were  10,  12,  14,  or  16  point  depending  on  the  treatment  group. 

Procedure 

The  subjects  received  the  following  instructions  before  reading  the  first  passages:  (a)  position  themselves  to  a 
height  with  their  eyes  between  the  middle  of  the  screen  and  the  top  of  the  screen,  (b)  seat  themselves  between  24"  and  30" 
from  the  screen,  and  (c)  enter  their  year  of  birth  and  await  further  instructions.  After  each  student  completed  these  tasks 
they  were  instructed  about  the  passage  content,  number  of  questions,  and  the  15  minute  time  limit. 
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The  subjects  proceeded  through  the  reading  comprehension  test  at  their  own  pace.  The  size  of  the  text  on  the  screen 
was  randomly  assigned  by  the  computer  program.  Upon  completion  of  the  testt  the  subjects  responded  to  a question 
about  font  size  satisfaction.  If  unsatisfied,  the  subjects  chose  a font  size  front  10,  12,  14,  and  16  point  size. 

The  subjects  were  observed  to  determine  those  individuals  not  within  the  range  of  24"  to  30"  from  the  screen. 
These  subjects  who  were  not  within  the  range  were  noted  in  the  experimental  log. 

Results 

Dependent  measures 

There  are  four  dependent  measures:  reading  comprehension,  satisfaction,  preference,  and  distance  from  computer 
screen.  Reading  comprehension  measured  participants'  reading  comprehension  of  test  passages.  Satisfaction  measured 
participants'  satisfaction  with  assigned  font  size.  Preference  measured  participants'  preferred  font  size  after  the  treatment. 
Posture  indicates  whether  the  students  were  inside  or  outside  the  optimal  2-2  1/2  feet  from  the  screen  during  the  reading 
of  the  passages. 

Independent  Measures 

There  are  three  independent  measures:  font  size,  age,  and  section.  Subjects  were  randomly  assigned  to  the  font  size 
groups  by  the  computer  program  and  were  assigned  sections  based  on  which  class  they  were  in. 

Design  and  data  analysis 

An  ANCOVA  was  conducted  using  the  dependent  variable  reading  comprehension  and  the  independent  variables  of 
font  size  and  section.  No  reading  comprehension  differences  were  found  for  the  main  effects  (F=0.35,  p=.78  for  font  size 
and  F=0.61,p=.77  for  section)  or  interaction  (F=1.23,  p=.23)  at  a = 0.05.  Likewise,  the  covariance  of  age  did  not  effect 
the  reading  comprehension  (F=0. 13,  p=.71).  The  ANCOVA  source  table  is  listed  in  Table  1. 


Table  1 

ANCOVA  for  Font  Size  bv  Section  with  Covariate  ojAge 


Source 

DF 

Sum  Of  Squares 

Mean  Square  F 

Font  Size 

3 

21.64 

7.21 

0.35 

Section 

8 

98.70 

12.34 

0.61 

Font  Size  by  Section 

21 

524.68 

24.98 

1.23 

Age 

1 

2.67 

2.67 

0.13 

Error 

127 

2584.52 

20.35 

Total 

160 

3297.98 

A Chi-square  was  performed  to  examine  whether  a significant  difference  occurred  between  font  size  and  satisfaction. 
A significant  difference,  shown  in  Table  2 (Chi-square  = 74.204,  p c.001)  was  found  for  satisfaction  at  a =.05. 


Table  2 


Chi-Square  of  Font  Size  by  Satisfaction 


Font  Size 

No 

Satisfaction 

Yes 

Total 

Chi-square 

10 

44 

1 

45 

74.204 

97.78 

2.22 

12 

31 

6 

37 

. - 

83.78 

16.22 

14 

22 

25 

47 

46.81 

53.19 

16 

3 

29 

32 

9.38 

90.63 

Total 

100 

61 

161 

62.11 

37.89 

100.00 

A Chi-square  was  performed  to  examine  whether  a significant  difference  occurred  between  font  size  and  preference 
(SeeTable  3).  A significant  difference  (Chi-square  = 51.81,  p <.00 1)  was  found  for  preference  at  a =.05. 
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Table  3 

Chi-sauare  of  Font  Size  bv  Preference 

Font  Size 

10 

Preference 

12  14 

16 

Total 

Chi-square 

10 

i 

0 

24 

20 

45 

51.81 

2.22 

0.00 

53.33 

44.44 

12 

0 

6 

14 

17 

37 

0.00 

16.22 

37.84 

45.95 

14 

0 

0 

27 

20 

47 

0.00 

0.00 

57.45 

42.55 

16 

0 

0 

1 

31 

32 

0.00 

0.00 

3.13 

96.88 

Total 

1 

6 

66 

88 

161 

0.62 

3.73 

40.99 

54.66 

100.00 

A Chi-square  was  performed  to  examine  whether  a significant  difference  occurred  between  subject’s  distance  from 
screen  and  font  size.  No  significant  difference  (Chi-square  « 4.78,  p = 0.573)  was  found  at  a =.05.  See  Table  4 for  the 
Chi-square  results. 


Table  4 

Chi-square  of  font  size  bv  distance  from  screen 


Distance 


Font  Size 

Leaning 

Forward 

Leaning 

Normal 

Back 

Total  Chi-square 

10 

14 

31 

0 

45  4.78 

31.11 

68.89 

0.00 

12 

8 

28 

1 

37 

21.62 

75.68 

2.70 

14 

16 

29 

2 

47 

34.04 

61.70 

4.26 

16 

10 

22 

0 

32 

31.25 

68.75 

0.00 

Total 

48 

110 

3 

161 

29.81 

68.32 

1.86 

100.00 

Discussion 

The  research  hypothesis  that  an  optimal  font  size  for  CBI  design  could  be  determined  by  our  efforts  lacked  support. 
Reading  comprehension  scores  did  not  show  a significant  difference.  Most  subjects  needed  more  than  the  allowed  fifteen 
minutes  to  complete  the  passages. 

Results  indicated  a lack  of  satisfaction  with  the  assigned  font  sizes  of  10  and  12  points,  however  they  were  satisfied 
with  font  sizes  of  14  and  16  points.  Subjects  preferred  the  16  point  size  to  the  other  font  sizes.  This  resuit  indicates  a 
possible  ceiling  effect  for  font  size  at  16  points.  From  this  we  may  infer  that  optimal  font  size  recommendations  of  10  - 
12  point  derived  from  print  media  research  (Hooper  & Hannafin,  1986;  Tinker,  1963)  are  not  adequate  for  CBI 
applications.  Further  our  results  conflict  with  those  of  Heines  (1984)  and  Criswell  (1989)  which  recommend  a font  size 
of  10  to  12  points.  However,  our  results  do  agree  with  those  of  Pastoor,  Schwarz  & Beldie  (1983)  and  Bailey  and 
Milheim  (1991)  which  favor  larger  font  sizes. 

This  study  had  a number  of  limitations.  The  first  group  of  subjects  all  received  the  10  point  font.  This  unexpected 
result  apparently  occurred  as  a consequence  of  the  way  HyperCard  began  its  random  number  generation  function.  Subjects 
could  have  been  randomly  assigned  in  advance  and  not  randomly  assigned  by  group  from  the  HyperCard  random  number 
generator.  Legibility  could  have  been  more  closely  controlled.  The  reading  test  was  expected  to  be  completed  within  15 
minutes.  However,  only  8%  of  the  subjects  completed  all  passages  and  questions.  Subjects  should  have  been  allowed 
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more  time  to  complete  the  passages  and  questions  or  fewer  passages  and  questions  should  have  been  assigned.  This  study 
did  not  take  into  account  other  font  styles  or  the  use  of  double  spacing. 

The  results  of  this  study  have  two  primary  implications.  Future  research  is  needed  because  font  size 
recommendations  based  on  print  media  are  inadequate,  and  the  optimal  size  font  for  CBI  applications  is  still  not  known. 
Future  research  should  include  the  use  of  larger  font  sizes  and  take  the  above  limitations  into  account. 
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Introduction 


The  evolution  of  information  delivery  systems  on  the  Internet  into  World  Wide  Web  "browsers"  like  Netscape  or  Mosaic 
presents  an  exciting  field  for  study  in  education  as  well  as  in  business.  Such  tools  are  rapidly  becoming  a major  interest 
to  small  and  large  business  concerns  (Ellsworth,  1995)  and  may  soon  impact  the  nature  of  education  in  profound  ways 
(Perelman,  1992).  Compared  with  text-based  information  system  such  as  Gopher  or  Veronica,  new  graphical  interface  is 
one  of  the  predominant  features  in  the  World-Wide  Web.  Everyday,  hundreds  of  new  graphics  with  new  visual  effects 
appear  on  the  Web— a picture  is  worth  a thousand  words.  However,  Web  designers  should  remember  that  a picture  takes 
a thousand  times  more  data  than  a word,  which  means  it  takes  much  longer  to  transfer,  and  often  viewers  are  not  patient 
enough  to  wait  for  the  picture  to  load.  This  paper  focuses  on  the  image  map,  a key  element  in  these  new  graphical 
interfaces,  and  the  reactions  of  its  viewers.  Understanding  the  visual  characteristics  of  the  image  map  is  critical  to  the 
design  of  effective  information  display  and  access  tools.  This  study  investigates  nine  different  image  maps  selected  from 
current  home  pages  on  the  World  Wide  Web.  These  image  maps  were  evaluated  regarding  their  effectiveness  by 
independent  viewers  and  using  criteria  relating  to  visual  literacy  and  hypermedia  design.  Research  results  are  then 
discussed  relative  to  visual,  navigational,  and  practical  characteristics. 

World  Wide  Web 

As  recently  as  the  middle  of  1993,  the  delivery  of  hypermedia  modules  over  computer  networks  was  problematic.  A 
mechanism  was  needed  that  would  standardize  structural  principles  for  the  delivery  of  such  modules  and  that  would  be 
"within  the  economic  reach  of  ordinary  users”  (Howard,  1993).  By  the  middle  of  1994,  exactly  such  a mechanism,  the 
World  Wide  Web,  had  become  available.  Now,  the  full  spectrum  of  communication  tools  is  available  electronically. 
Today  many  resources  of  hypermedia-integrated  text,  graphics,  audio,  animation,  and  video  are  accessible  across  the  World 
Wide  Web  (WWW). 

The  WWW  is  the  fastest  growing  information  tool  on  the  Internet  (Descy,  1994).  Along  with  the  rich  multimedia 
communications,  graphical  web  browsers  also  provide  a very  high  level  of  interaction.  Users  can  quickly  jump  to  new, 
related  information  using  interactive  linking  tools  like  “buttons,"  "hot  words,"  or  "image  maps.  These  hyperlinks  can 
connect  users  to  new  information  resources  on  their  own  campus,  within  their  own  towns,  or  around  the  world  with  the 
simple  click  of  a mouse  button  (Dougherty  & Kom  1994). 

Image  Maps 

Graphics  may  be  used  to  capture  a viewer's  attention,  to  hold  interest  (Grabinger,  1993),  to  supplement  and  reinforce 
textual  materials  (Lucas,  1991)  or  to  provide  organizational  overviews  of  complex  data  (Grabinger,  1993),  (Koneman  & 
Jonassen,  1994).  Likewise,  in  web  browsers,  graphics  may  just  add  visual  interest,  mark  a simple  link  to  related 
information,  or  indicate  organized  multiple  links  to  a complex  information  set.  In  the  WWW,  these  links  are  called  . 
hyperlinks. 

A hyperlink  is  the  connection  between  a word  or  graphic  in  an  active  browser  view  to  another  file  anywhere  on  the  World 
Wide  Web  (Hudak-David,  1994).  A simple  graphic  hyperlink  connects  a single  image  to  a single  information  resource 
(Figure  1).  The  image  map  (Figure  2)  provides  easy  multiple  choice  access  for  web  users.  An  image  map  connects  a 
single,  complex  visual  by  multiple  hyperlinks  to  a number  of  related  information  resources  (Wiggins,  1994).  A web 
browser  recognizes  when  a user  clicks  a displayed  "hot  word"  or  "hot  spot"  and  simply  opens  a connection  to  the  pre- 
programmed uniform  resource  locator  (URL  - the  Internet  address  of  the  desired  resource).  Currently,  the  image  map  is 
the  most  refined  expression  of  this  integration. 
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Figure  1.  Simple  Graphic 
Links  to  Single  Data  Sets. 
(http://vinny.csd.mu.edu/ 
-howard/monet.html) 


Figure  2.  Multiple  Links 
From  One  Image  Map  to 
Several  Data  Sets. 
(http://home.netscape.com) 


Image  Map  Design  Issues 


Effective  image  maps  require  careful  design  with  respect  to  visual  and  interactive  aspects  as  they  pertain  to  the  users. 

User  interface  design  for  image  maps  touches  on  three  broad  areas:  visual  characteristics,  navigational  characteristics,  and 
practical  characteristics. 

Visual  Characteristics 

Visually,  as  a graphic,  an  image  map  should  be  appealing.  At  the  same  time,  it  is  important  that  an  image  map  be 
easily  recognized  as  a linking  tool.  There  should  be  visual  cues  that  tell  users  they  are  dealing  with  an  image  map.  The 
visual  content  presented  in  an  image  map  should  also  support  a typical  viewer's  expectations  relative  to  the  information 
the  graphic  is  supposed  to  represent  (Lucas,  1991).  For  example,  an  image  map  for  a university  in  Arizona  would 
probably  not  be  designed  around  an  arctic  theme. 

The  individual  graphics  and  related  "hot  spots"  within  an  image  map  should  represent  the  natures  of  the  key 
informational  segments.  For  instance,  a book  icon  is  a good  representation  for  library  services.  Hot  spots  within  an 
image  map  should  be  easy  to  recognize  (Lucas,  1991;  Jones,  1995).  Often,  a change  in  the  mouse  cursor  will  cue  the 
presence  of  hot  spots.  In  other  cases,  maps  or  similar  structural  features  suggest  the  possibilities  of  hot  spots.  An 
image  map  should  be  artistic  and  visually  appealing  but  not  too  busy  or  cluttered  (Lucas,  1991 ; Jones,  1995;  Grabinger, 

1993) . 

Navigational  Characteristics 

Navigation  is  the  act  of  recognizing  and  initiating  a hyperlink  to  new  information.  Navigational  aids  should,  if  desired 
by  the  viewer,  permit  returning  to  the  starting  point  in  order  to  investigate  other  possibilities.  Back  tracking  and  book 
marking  are  additional  navigational  aids  to  help  clarify  the  structure  of  multiple  paths  and  provide  quick  repeated  accesses 
to  pertinent  information  (Desberg,  1994;  Jones,  1995).  Image  maps  should  help  users  build  mental  models  of  the 
underlying  structure  and  inter-relationships  of  information  resources  (Lucas,  1991). 

Navigational  pit  falls  such  as  dead  ends,  endless  choices,  or  links  to  non-existent  resources  should  be  avoided.  Links  to 
non-existent  resources  are  troublesome  given  how  frequently  web  pages  change.  Finally,  "help"  should  be  available  on 
demand  or  when  "non-hot-spot"  areas  of  an  image  map  are  clicked  (Desberg,  1994). 

Practical  Characteristics 

Practical  concerns  consider  the  implementation  or  working  characteristics  of  image  maps.  An  image  map  graphic  should 
not  take  too  long  to  load  (Desberg,  1994)  and  the  special  effects  (image  overlay,  fades,  fonts,  extraneous  visual 
information,  colors,  3D,  animations,  video,  backgrounds)  should  not  clutter  the  display  or  frustrate  the  viewer  (Desberg, 

1994) .  It  should  be  easy  to  tell  what  resources  have  been  visited  in  the  current  session  with  the  image  map  (Jones, 

1995) .  A visual  consistency  (Grabinger,  1993)  should  tie  the  image  map  and  its  related  information  resources  together, 
cueing  the  viewer  that  he  or  she  has  not  branched  off  into  unknown  or  unexpected  territories  (Lucas,  1991).  The  graphics 
and  the  related  information  resources  should  fit  the  target  audience  (Desberg,  1994). 

Methodology 

Development  of  Survey  Questionnaire 

Since  there  is  no  previous  research  about  image  maps,  this  research  represents  a pilot  study  directed  toward  developing  an 
evaluation  procedure.  Starting  with  guidelines  from  several  computer-related  product  checklists  in  the  literature  (Lucas, 
1991;  Desberg,  1994;  Cates,  1992;  Tolhurst,  1992),  the  authors  developed  a 20  question  survey  form  addressing  design 
issues  based  on  visual,  navigational,  and  practical  characteristics.  Two  initial  surveys,  conducted  as  a pilot  for  formative 
evaluation,  determined  changes  and  clarifications  needed  in  the  questionnaire.  Following  the  pilot,  the  scale  of  evaluation 
was  changed  from  1-4  (strongly  disagree  to  strongly  agree)  to  1-10  (ten  step  scale  from  disagree  to  agree)  in  order  to 
clarify  and  strengthen  the  differences  in  viewer  opinions. 

Initially,  questions  were  stated  so  that  a high  numeric  response  indicated  strong  agreement.  However,  some  questions 
became  awkward  to  read  in  this  format.  For  example,  one  question  stated  "I  was  not  frustrated  by  the  image  map."  The 
authors  were  concerned  the  question  could  be  confusing  because  of  the  non-typical  negative  structure  so  a few  questions 
were  re-phrased  on  the  survey  form.  This  sample  question  was  re-written  "I  was  frustrated  by  the  image  map."  In  this 
format,  a response  of  1 suggests  a good  image  map. 

Of  the  twenty  questions,  eight  questions  covered  visual  characteristics.  For  example,  the  questions  ask  how  easy  the 
graphic  is  to  recognize  as  an  image  map,  how  noticeable  the  hot  spots  are,  or  how  appealing  is  the  image  map.  The 
navigational  aspects,  using  seven  questions,  ask  how  easy  it  is  to  move  from  page  to  page,  how  easy  it  is  to  return  to 
the  starting  point,  or  if  help  is  available.  Five  questions  cover  the  practical  concerns  such  as  how  long  graphics  take  to 
load,  how  consistent  the  graphics  are  at  a particular  site,  or  if  the  image  maps  are  frustrating. 
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Selection  of  WWW  Sites 

The  evaluation  scope  was  limited  to  educational  or  institutional  home  pages  on  the  WWW.  The  Net  Directory  option  of 
the  web  browser  Netscape  2.0  was  the  sampling  source  using  the  category  of  education.  Two  sites  from  each  of  36  sub- 
categories of  the  educational  area  were  randomly  selected  by  the  authors.  While  examining  the  72  site  options,  the 
authors  selected  image  maps  that  covered  the  spectrum  from  difficult  to  facilitative  for  typical  users.  Nine  sites,  (Figures 
3 through  11)  representing  visual,  navigational,  and  practical  image  map  characteristics,  were  selected  for  the  final 
evaluation  by  independent  viewers. 
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Figure  3.  Discovery  Channel 

(http://www.discovery.com/DCO/ 

doc/1012/online.html) 


Figure  4.  Educom  Image  Map 
(http://www.educom.edu/) 


Figure  5.  ExCite  Image  Map 
(http://oberon.educ.sfu.ca:80/ 
newhome.html) 


Figure  6.  Map  of  Monument  and 
Museums  of  Paris 
(http://meteora.uscd.edu/~norman/ 
paris/Maps/MM) 


Figure  7.  QuickTime 

(http://qt  vr3  .quicktime.apple.com/) 


Figure  9.  UCLA  Image  Map 
(http://www.ucla.edu/) 


Figure  8.  Texas  A&M  Campus  Map 
(http://www.tamu.edu/test/map/map.cgi) 


Figure  10.  UWYO  Image  Map 
(http://www.uwyo.edu/) 
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Figure  11.  Virtual  Tourist 
(http://wings.buffalo.edu/world) 

Independent  Viewers 

Nine  viewers  were  involved  in  this  survey.  Most  of  them  were  graduate  students  taking  instructional  technology  classes. 
Others  were  volunteers.  They  individually  came  to  a computer  station  connected  to  the  Internet  and  spent  about  one  hour 
evaluating  the  image  maps.  Addresses  of  the  nine  sites  and  the  survey  form  were  given  to  the  viewers  and  they  usually 
decided  which  sites  to  evaluate  out  of  the  nine  possibilities.  Viewers'  activities  were  observed  by  the  authors  during  their 
evaluation  time.  Only  two  viewers  finished  evaluating  all  nine  sites  within  an  hour  while  one  individual  spent  an  hour 
and  a half.  Most  of  them  evaluated  four  or  five  sites  within  an  hour.  Those  who  finished  all  nine  sites  within  an  hour 
were  already  familiar  with  the  World  Wide  Web  and  image  maps.  Most  viewers  were  quite  new  to  the  WWW  and  spent 
the  first  ten  to  fifteen  minutes  exploring.  Those  who  were  unfamiliar  with  the  WWW  were  given  some  advice  while 
exploring  the  image  maps. 


Results  and  Conclusions 

Fifty  surveys  were  collected  from  nine  viewers.  Each  site  was  evaluated  four  to  seven  times  by  different  viewers.  For 
analysis,  every  question  item  has  the  same  value  system:  the  higher  the  score,  the  more  positive  the  map  characteristic. 
The  20  survey  items  were  grouped  into  three  categories:  visual,  navigational,  and  practical.  Because  each  category 
reflects  a different  number  of  questions  and  a different  number  of  viewer  evaluations,  the  total  score  of  each  category  was 
divided  by  the  number  of  questions  times  the  number  of  viewers  in  order  to  give  a comparable  weight  to  each  category. 
Figure  12  shows  the  analysis  of  the  data. 


Figure  12.  Comparison  of  Average  Viewer  Ratings  by  Site: 
Overall,  Visual,  Navigational,  and  Practical. 


Overall  Rankings 

Overall,  the  image  maps  of  UCLA  (Figure  9)  and  Educom  (Figure  4)  were  selected  as  the  best.  In  visual 
characteristics,  UCLA  and  Educom  also  claim  the  top  two  rankings.  From  a navigational  perspective,  the  Virtual 
Tourist  (Figure  1 1),  the  Texas  A&M  campus  map  (Figure  8),  and  the  Educom  image  map  occupy  the  top  three  spots  in  a 
very  close  ranking.  Finally,  from  a practical  viewpoint,  UW  (Figure  10)  has  the  highest  ranking.  The  UW  result  is 
supported  by  an  independent  study  of  nearly  500  college  web  pages  conducted  at  Columbia  University,  which  also 
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considered  visual  and  practical  characteristics  (Johnson,  et.  al.,  1995).  Their  study  highly  rated  the  UW  web  page  for 
good  presentation  of  general  information. 

Visual  Perspective 

There  seems  to  be  two  conflicting  elements  at  work  in  the  visual  analysis.  Image  maps  can  be  very  artistic  and  complex 
or  rather  simple.  When  viewers  are  looking  for  direction  to  information,  they  appear  to  prefer  simple  image  maps.  Even 
when  viewers  were  asked  how  visually  appealing  a graphic  was,  they  seemed  to  judge  the  degree  of  appeal  by  how  easily 
they  could  get  to  information.  If  they  could  quickly  locate  hot  spots  in  the  image  map  and  if  the  icons  accurately 
represented  the  information,  they  seemed  to  prefer  the  graphic.  The  UCLA  (Figure  9)  and  Educom  (Figure  4)  image 

maps,  which  ranked  the  highest,  both  use  a simple  and  well-grouped  layout. 

In  Grabinger's  study  (1993),  simplicity  is  one  of  the  important  factors  in  computer  screen  design.  Besides  simplicity, 
Lucas  (1991),  in  a study  about  effective  computer-learner  interfaces,  states  that  grouping  has  a very  strong  influence  on 
commanding  and  focusing  viewer  attention.  As  counter-examples,  Quicktime  (Figure  17),  Excite  (Figure  5),  and 
Discovery  (Figure  3)  present  more  artistically  interesting  image  maps.  However,  these  three  all  violate  the  rules  of 
simplicity  and  united  grouping.  For  instance,  the  Quicktime  image  map  scatters  hot  spots  all  around  the  graphic  which 
de-focuses  or  scatters  viewers'  attention  and  results  in  the  lowest  visual  ranking.  The  Discovery  and  Excite  image  maps 
appear  to  override  marginal  grouping  with  a too  complex  image. 

Navigational  Perspective 

Navigation  deals  with  the  mechanics  of  moving  through  the  available  information.  Regarding  navigational  aspects,  the 
Virtual  Tourist  (Figure  11),  Texas  A&M  campus  (Figure  8),  and  Educom  (Figure  4)  image  maps  were  selected  the  best. 
These  sites  do  not  give  the  viewers  many  layers  of  choices.  Thus,  it  is  easy  to  link  out  in  a single  step  and  quickly 
return  to  the  starting  point.  The  high  ratings  for  the  Virtual  Tourist  and  Texas  A&M  image  maps  may  also  be  a 
function  of  the  physical  nature  of  the  information  they  represent.  Grabinger  (1991)  suggests  that  viewers  prefer  screens 
designed  to  closely  reflect  the  content  of  the  subject  matter. 

Practical  Perspective 

As  to  practical  aspects,  UW  (Figure  10),  Educom  (Figure  4),  and  UCLA  (Figure  9)  ranked  highest.  Typically,  these 
image  maps  are  smaller  and  load  quickly.  Special  effects  are  minimal.  Consistency  is  a strong  characteristic  of  these 
image  maps  and  their  related  information  pages.  The  linking  pages  have  visual  cues  such  as  icons  or  graphics  very 
similar  to  the  original  image  map.  These  maps  were  also  highly  rated  in  visual  criteria,  suggesting  practical 
characteristics  may  be  correlated  with  visual  characteristics.  Artistic  images,  such  as  the  Discovery  or  QuickTime  image 
maps  (Figure  3,  Figure  7),  tend  to  be  large,  complex  and  slow  to  load. 

Discussion  and  Recommendations 

This  initial  study  presents  a number  of  factors  that  appear  to  be  important  to  effective  image  map  design  when  a viewer's 
purpose  is  to  locate  specific  information.  In  this  instance,  functionality  is  the  most  important  aspect.  Simple  graphics 
that  load  quickly  and  clearly  designate  hot  spots  appear  to  be  preferred  by  the  viewers  in  this  study.  Viewers,  particularly 
novices,  also  wanted  some  site-specific  help  available  when  navigating  web  sites.  Figure  13  presents  the  authors 
recommendations  as  a "quick  guide"  for  designing  effective  image  maps  for  informational  purposes.  The  Educom  (Figure 
4)  image  map,  which  placed  somewhere  in  the  top  three  rankings  in  each  category,  is  the  best  example  of  these 
recommendations. 

There  may  be  different  design  guidelines  when  entertainment  is  the  purpose  of  the  image  map.  There  is  an  interesting 
tension  between  artistic  and  functional  aspects  of  image  map  design.  A web  designer  must  consider  the  balance  between 
the  purposes  served  by  the  maps  and  the  intentions  of  the  prospective  viewers.  From  this  research,  viewers  seeking 
information  want  functional,  simple  maps  but  viewers  "surfing  the  web"  may  pass  over  such  simple  maps  for  the  more 
artistic  and  entertaining  images.  The  challenge  to  the  image  map  designer  is  to  find  the  balance  between  artistic  needs  to 
catch  the  viewer  and  functionality  to  allow  the  viewer  to  find  information  quickly.  This  balance  issue  presents 
opportunities  for  future  research. 
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“QUICK  Guide” 
for  Functional  Image  Maps 

V Use  simple  graphics 

V Use  smaller  than  larger  image  sizes 

V Clearly  define  hot  spots 

V Use  unified  rather  than  scattered  grouping 

V Choose  icons  to  accurately  represent  information 

V Design  image  map  to  model  the  structure  of  information 

V Use  a minimal  number  of  layers 

V Limit  the  number  of  choices 

V Include  site-specific  help 

Figure  13.  Summary  Design  Guide  for  Functional  Image  Maps 
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"The  soul  never  thinks  without  a picture. " Aristotle  (390  BC) 


Since  before  recorded  history , when  people  were  sitting  around,  staring  into  the  fire  or  relating  their  experiences,  their 
mind  would  visualize  thoughts  or  stories  in  terms  that  could  be  understood  and  processed.  The  human  mind  has  always 
converted  information  into  visual  images  to  be  encoded  and  stored  in  long  term  memory.  People  have  learned  primarily 
through  visualizing  since  the  time  of  evolution  (Psotka,  1994). 

For  instance,  when  reading  or  hearing  the  word  “pencil”,  one  creates  a mental  image  of  a pencil.  With  the  advent  of 
virtual  reality  (VR)  and  virtual  environments  (VE),  images  play  an  even  more  important  role  within  the  realm  of 
visualization  by  generating  .and  immersing  the  user  in  both  visual  images  and  auditory  messages.  Seen  through  VR,  the 
pencil  once  visualized,  comes  “alive”  in  many  different  ways,  thus  the  visualization  experienced  becomes  enhanced  many 
times  over. 

Virtual  Reality 

Virtual  Reality  is  an  exciting  new  medium  based  on  a concept  which  allows  total  stimulation  of  one's  senses  through 
using  human/computer  interfaces.  It  already  has  many  applications  in  training  simulators,  nano-science,  medicine, 
entertainment,  electronic  technology,  and  manufacturing.  In  the  near  future,  it  will  have  increasingly  greater  applications 
which  are  bound  to  impact  all  aspects  of  life.  Its  effect  on  society,  through  both  what  is  developed  through  advanced 
technology  and  the  very  essence  of  Virtual  Reality,  in  and  of  itself,  is  likely  to  cause  a cultural  change  in  society  as  did 
the  motion  picture  or  television  industry. 

Virtual  Reality  is  still  in  its  infancy  and  the  ideal  usability  of  this  technology  is  sometime  in  the  future.  When 
comparing  Virtual  Reality  with  a medium  such  as  television,  one  has  to  realize  even  though  TV  is  only  two 
dimensional  and  despite  its  perceived  simplicity,  it  has  an  extremely  strong  immersive  quality  and  influence  over  people, 
especially  adolescence  (i.e.  the  issue  of  television  violence). 

Virtual  reality  is  designed  to  be  as  close  to  true  three  dimensional  as  possible  and  its  programs  require  interaction  from 
users,  not  just  immersion.  The  effects  this  will  have  on  individuals  and  society  as  a whole  cannot  be  totally  foreseen  at 
this  time.  But,  the  implications  of  what  effects  it  can  have  are  beginning  to  surface  and  should  be  addressed  while  the 
technology  is  still  in  its  infancy. 


It  has  long  been  known  that  we  perceive  and  leant  through  a variety  of  different  mediums.  Aristotle,  in  350  BC,  wrote 
about  mediums  used  in  perception: 


There  has  to  be  a naturally  attached  medium...  The  following  problem  might  be  raised.  Let  us  assume  that  every  body 
has  depth,  i.e.  has  three  dimensions,  and  that  if  two  bodies  have  a third  body  between  them  (environment)  they  cannot  be 
in  contact  with  one  another...  The  problem,  then  is:  does  the  perception  of  all  objects  of  sense  take  place  in  the  same 
way,  or  does  it  not,  e.g.  taste  and  touch  requiring  contact  (as  they  are  commonly  thought  to  do),  while  all  other  senses 
perceive  over  a distance?  The  distinction  is  unsound;  We  perceive  what  is  hard  or  soft,  as  well  as  objects  of  hearing, 
sight,  and  smell,  through  a ‘medium',  only  that  the  latter  are  perceived  over  a greater  distance  than  the  former;  that  is 
why  the  facts  escape  our  notice.  For  we  do  perceive  everything  through  a medium;  but  in  these  cases  the  fact  escapes 
us. ..we  perceive  because  the  medium  produces  a certain  effect  on  us  (Stevenson,  1995). 

Thus,  if  our  senses  trick  us  in  the  real  world,  how  far  then  can  we  be  additionally  “tricked”  by  this  new  medium  without 
demonstrating  some  type  adverse  reaction  to  it.  An  example  again  would  be  television.  It  portrays  motion  , but  in 
actuality,  it  is  nothing  more  than  a sequence  of  scanned  lines  presented  in  a manner  as  to  create  an  illusion  of  motion. 
Coupled  with  (his,  is  an  illusion  of  depth  by  the  placement  of  graphical  objects  which  allows  the  watcher  to  feel 
“presence''  or  “telepresence”  (a  sense  of  being  in  another  place). 
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The  Physio-Psychological  Aspects 

Virtual  reality  or  virtual  environments  are  oriented  primarily  as  a spatial  and  temporal  medium.  Once  immersed  in  it, 

“our  minds  construct  a closure  to  create  the  experience  of  inclusion”  (Bricken  and  Geoffrey,  1994)”.  The  VR  industry  is 
trying  to  achieve  a true  four  dimensional  environment  (height,  width,  depth,  and  motion)  that  will  assist  in  the  functions 
of  human  performance  on  many  levels.  Even  though  touch/feel,  sound  and  smell  are  being  incorporated  into  this 
technology,  the  primary  focus  is  on  the  visual  sense.  This  dominant  visual  property  of  VE’s  requires  the  user  to  leant 
and  to  relate  through  cognitive,  psychomotor,  and  affective  processes  that  are  relative  to  but  dissimilar  to  the  processes 
used  in  the  real  world,  (i.e.:  flying  over  rooms,  walking  through  walls,  lifting  objects  that  are  too  small  or  too  large, 
maneuvering  in  space  while  repairing  equipment,  or  swimming  with  fish). 

There  is  greater  use  of  the  visual  senses  in  VR  than  in  almost  any  other  form  of  communication  with  “signing”  being  an 
exception.  The  very  act  of  signing  is  truly  four  dimensional,  totally  spatial  and  greatly  iconic.  Oliver  Sacks  in  his  book, 
Seeing  Voices,  addresses  the  fact  that  the  hearing  impaired  who  sign  have  better  perception  and  direction  of  motion  than 
other  people  of  their  same  age  and  abilities.  (Sacks,  1989).  This  is  most  likely  due  to  the  way  spatial  cues  are  processed 
along  with  experience  in  processing  in  this  manner. 

In  the  virtual  environment  a user  is  able  to  accomplish  normally  unrealistic  or  supernatural  tasks  along  with  normal 
tasks.  Uniquely,  the  larger  numbers  of  these  tasks  are  represented  in  the  form  of  objects  and  /or  icons,  and  they  are 
manipulated  with  the  movement  of  hands  and  fingers.  This  presents  a similarity  between  signing  and  virtual 
environments.  Perhaps  the  spatial  and  temporal  processes  that  arc  used  by  the  hearing  impaired  can  be  used  assist  a user 
in  VR  to  adapt  to  uncommon  events  that,  prior  to  entering  the  virtual  environment,  could  not  or  did  not  exist,  how  an 
event  or  task  in  a virtual  environment  processed  mentally  and,  in  what  manner  in  which  it  affects  the  user’s  emotions. 

Another  area  of  the  VE  is  the  spatial  process  the  brain  uses  to  encode  and  decode  visual  information  received,  what 
signals  are  actually  received,  and  the  response  returned.  In  the  Soviet  Union  it  was  not  an  uncommon  practice  to  remove 
healthy  eyes  from  blind  people  and  use  the  eyes  to  restore  other  people’s  vision,  which  seems  on  the  surface  to  be  an 
acceptable  practice.  However,  over  a period  of  time  it  was  noticed  that  many  of  the  people  who  were  blind,  developed 
sleep  disorders.  It  was  then  determined  that  these  people  could  not  see  in  the  normal  light  wave  frequencies,  that  the  brain 
had  been  able  to  determine  day  and  night  through  other  frequencies  or  mediums  being  transmitted  through  the  eyes  before 
they  had  been  removed.  It  could  very  well  be  that  there  arc  certain  optical  or  near-optical  stimuli  that. are  present  in  VE’s 
that  may  have  less  than  positive  influence  on  the  user’s  performance. 

Technology  Transfer 

The  human  brain  processes  information  in  a manner  we  don’t  fully  understand  at  present.  Physiologically  we  have  yet  to 
determine  how,  as  well  as  where,  different  information  is  processed  and  stored  in  the  brain.  Visual  cortical  neurons  that 
are  sensitive  to  motion  direction,  vertical  axis,  horizontal  axis,  or  opposing  directions,  and  the  method  in  which 
processing  of  the  information  occurs  are  examples  of  visual  perception  and  processing  questions  yet  to  be  answered.  In 
the  normal  world,  we  orient  ourselves  to  binocularly  perceive  the  size  and  distance  of  an  object,  using  pre-established 
mental  conditioning  (learned  experience).  We  process  this  environmental  information  without  consciously  realizing  what 
or  how  we  do  this.  From  their  research  on  field  of  view  (FOV)  and  other  related  areas,  Joseph  Psotka,  US  Army 
Research  Institute,  Sharon  A.  Davison,  Catholic  University,  and  Sonya  A.  Lewis,  Howard  University  describe  the 
problem: 

The  visual  space  has  to  be  coordinated  with  the  proprioceptive  space  of  the  eyes,  head  and  hands.  This  has  not  always 
coordinated  in  virtual  environments,  nor  is  it  easy  to  engineer,  since  the  psychological  cues  for  these  spaces  remain 
complex  and  poorly  understood  (Psotka.  J.,  Davison,  S.,  and  Lewis,  S.A.,  1993). 

Since  virtual  reality  largely  depends  on  spatial  and  temporal  recognition,  the  proper  positioning  of  “objects”  within  a 
virtual  environment  is  crucial  to  achieving  any  degree  of  reality.  In  essence,  the  human  mind  is  being  tricked  into  its 
perception  of  the  “objects”  and  their  placement  within  VEs.  Any  three  dimensional  “object”  is  perceived  to  have  six 
points  of  reference  or  centers  of  origin;  i.e.  a cube  has  a front,  back,  two  sides,  top  and  bottom.  In  order  to  determine 
the  size  and  distance,  there  has  to  be  a seventh  point  considered  to  be  the  “oneegocenter”  [or  central  ego  center], 
"cgocenter”,  or  individual  view  point. 

Once  all  six  reference  points  have  been  established,  then  a perceived  size  and  distance  can  be  assigned  to  that  “object”. 
This  completed  information  is  then  encoded  and  stored  in  the  appropriate  storage  area  of  the  brain.  From  there  this 
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process  is  repeated  with  any  other  “objects”  within  the  VE,  one  by  one  until  they  have  all  been  categorized  and  stored. 

At  this  point  the  next  scenario  is  processed  and  repeated  until  that  task  is  completed. 

Three  dimensional  motion  in  simulation  has  the  original  6 points  of  reference  that  are  moveable  (front-back,  up-down, 
left-right)  as  well  as  yaw  (angular  rotation  on  the  vertical  axis),  pitch  (movement  in  distance)  and  roll  (3-dimensional 
rotation  of  an  object).  These  are  known  as  the  “six  degrees  of  freedom”  of  movement.  The  seventh  point  or  the  point  of 
observation  is  also  required  to  perceive  and  define  the  object. 

Technology  vs.  Performance 

Virtual  environments  are  designed  in  a way  for  the  user  to  feel  “presence”  within  that  environment.  Presence  is  “...the 
suspension  of  disbelief  which  permits  us  to  share  the  digital  manifestations  of  fantasy  (Bricken  & Geoffrey,  1994).”  If 
the  VE  is  not  precisely  adequate  for  the  user,  then  there  can  be  adverse  effects  on  the  user.  For  instance,  if  the  binocular 
balance  (equal  recognition  between  both  eyes)  or  the  viewing  point  is  not  exact,  the  user  may  experience  motion 
sickness  or  other  manifestations  can  occur  (Psotka,  Davidson,  & Lewis,  1993).  If  the  proper  balance  between  the 
vestibular  and  visual  systems  are  not  met,  then  the  equilibrium  of  the  human  autonomic  system  can  produce  undesired 
symptoms  such  as  dizziness,  upset  stomach,  headaches  etc.  Even  certain  frequencies  of  sound  can  produce  undesired 
effects,  with  some  known  to  trigger  epileptic  seizures.  Bruce  Gil  of  Viewpoint  Datalabs  International  stated  that  a 
warning  notice  is  placed  on  certain  VR  arcade  games  to  alert  people  with  epileptic  tendencies  (personal  communication, 
July  1994). 

Among  the  known  affects  reported  on  this  new  human  computer  interface  medium,  is  the  possibility  of  flashbacks  from 
certain  “flight  simulators”  using  HMD  (head  mounted  displays)  that  produce  strong  effects  as  to  cause  the  restriction  on 
driving  or  flying  for  a period  of  time  (Isdale,  1993).  In  some  cases  “flashbacks”  occurring  some  hours  after  spending 
hours  in  a simulator  have  nearly  caused  fatal  automobile  accidents  (Office  of  Technology  Assessment,  1994).  Another 
study  by  the  Endinburgh  Department  of  Psychology  (Endinburgh  Virtual  Environment  Lab),  found  that  after  ten 
minutes  of  light  exercise  while  wearing  HMDs,  20  young  adults  displayed  clear  signs  of  binocular  stress.  Over  half  the 
subjects  also  reported  symptoms  of  visual  stress,  such  as  blurred  vision. 

Other  Problems  to  Address 

It  is  widely  believed  by  the  year  2000,  entertainment  will  dominate  the  largest  share  of  the  virtual  reality  market 
(Biocca,  1992).  Shapiro  and  McDonald  (1992)  express  concern  that  “heavy  VR  users”  with  less  skills  and  knowledge  in 
making  sophisticated  reality  judgments,  may  get  lost  in  the  VR  environment  The  very  essence  of  virtual  reality 
generates  responses  which  in  turn  stimulates  emotions.  Jaron  Lanier,  one  of  the  founders  of  VPL  Research  (the 
company  that  patented  the  data  glove)  relates  his  view  regarding  virtual  reality  games: 

Also,  there's  a problem  that  it's  (VR  technology)  sort  of  an  enforced  form  of  compulsive  behavior.  There  is  no  doubt 
that  all  of  us  - and  let's  say  adolescent  boys  in  particular  - go  through  a period  where  they  are  interested  in  killing 
things  and  aggression.  But  I think  the  point  is  if  they  do  it  in  the  play  ground,  it’s  part  of  a fluid  process  which 
continues  to  grow  and  change.  But,  if  they  get  caught  up  in  an  interactive  loop,  in  some  sort  of  a simulation 
entertainment  product,  then  they  get  stuck  in  and  relive  the  same  loop  again  and  again  (Shapiro  and  McDonald  p.  163- 
164). 

Virtual  environments  have  many  similarities  to  “in  depth  day  dreaming”.  There  is  inclusion,  immersion,  closure  and 
visual  images.  Comparatively  speaking,  if  day  dreaming  can  be  considered  as  an  escape  mechanism,  how  much  more 
intensive  and  in  what  manner  will  the  onset  of  virtual  reality  “games”  with  their  powerful  and  unnatural  environments 
affect  the  participants?  In  a recent  report  on  virtual  reality.  Daily,  Howard,  and  Caudell  (date  unknown)  express  their 
concerns  that  “...we  can  expect  the  full  potential  of  virtual  reality  to  be  compellingly  hypnotic,  especially  for 
entertainment  (p.23).” 

Issues  and  concerns: 

1)  Shapiro  and  McDonald  (1992)  point  out  that  research  has  not  focused  on  reality  judgments  in  virtual  reality  but 
instead  on  the  media  effects.  They  suggest  that  “Interacting  with  such  an  environment  may  often  tax  mental 
capabilities”,  and  the  more  real  the  environment  appears,  the  more  likely  “it  becomes  a me.ital  challenge  to  make 
judgments  about  reality  (p.  104).”  Just  individual  temperamental  personality  differences  determines  the  extent  of 
immersion  an  individual  person  experiences  (Psotka,  1994). 
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Cornell,  Bailey,  and  Bollet  (1994)  raise  concerns  related  to  the  well-being  of  unsuspecting  individuals  who  may 
experience  dramatic  and  highly-dangerous  side  effects  which  impact  mental  stability.  Is  there  danger  to  unsuspecting  users 
through  immersion  in  virtual  reality  and  other  similar  experiences?  What  effect  will  immersion  in  this  powerful  medium 
have  on  those  subseptable  to  schizophrenia,  bi-polar  disorders,  claustrophobia,  or  other  related  impairments? 

2)  There  seem  to  be  no  published  studies  in  the  combining  of  artificial  intelligence  (AI)  with  virtual  environments. 
Already  the  industry  has  termed  certain  AI  components  as  “V-actors”,  “V-extras”,  “V-experts”,  and  “V-agents”  to  be  used 
as  separate  “beings”  capable  of  changing  the  environment  within  VR  games  (Hamit,  1994).  These  “entities’”  purpose  is 
to  “work  together  to  undertake  tasks  beyond  the  capability  “of  the  players  or  any  one  entity.  What  effects  will  “entities” 
have  on  the  perception  and  judgment  of  users  in  virtual  environments  ? 

3)  Psotka,  Davidson,  & Lewis  (1993)  relate  an  experience  they  had  with  moving  a user  who  was  preoccupied  in  a VR 
“fish  world”  chasing  a fish.  They  moved  him  slightly  and  his  first  perception  was  that  it  was  done  by  an  invisible  force. 
They  continue  to  say  that  his  second  confusion  was  that  he  could  not  be  moved  in  VR  by  the  real  world.  They  observed 
that  the  subject’s  momentary  confusion  and  disorientation  lasted  several  seconds  before  he  could  orient  himself,  and  this 
was  in  a simple  scenario.  They  continue  by  adding  “Could  it  be,  that  with  practice  we  could  become  more  adept  at 
holding  these  two  realities  intact’  synchronically?  If  anything,  our  experiences  in  VR  are  making  it  easier  and  easier  to 
enter  VR  completely  immersed,  and  easier  to  forget  entirely  our  real  world  surroundings”  . 

What  will  be  the  long  term  effects  on  “heavy”  VR  or  simulator  users  in  relation  to  their  perception  and  interaction  with 
a real  environment?  If  just  after  a few  hours  of  “immersion”  in  a simulator  can  cause  serious  “flashbacks”,  how  much 
time  can  one  spend  in  a virtual  environment  without  psychological  effects? 

4)  We  must  consider  the  ramifications  of  Marshal  McLuhan’s  concept  of  the  message  being  the  medium  as  it  would 
apply  to  VR  and  VEs.  Virtual  Reality  (or  a virtual  environment)  is  an  extremely  powerful  psychological  medium.  Is 
Virtual  Reality  as  a medium  so  great,  that  it  overwhelms  the  training  or  task  when  it  is  being  presented  through  viitual 
environments? 

Conclusion 

As  with  any  new  technology,  there  are  uncertainties  of  its  potential,  uses,  and  control.  Virtual  reality  with  all  its 
magnitude  is  certainly  no  different.  As  in  other  technological  advances,  problems  will  arise  and  will  be  resolved. 
However,  it  was  the  intention  of  this  paper  to  focus  on  some  current  and  potential  problems  so  that  they  car.  be  studied 
and  resolved.  The  immersed  user  of  this  medium  and  any  undesired  effects  of  the  medium  should  be  the  primary  focus 
and  not  the  medium. 

It  is  our  belief  that  the  end  products  of  VR/VEs  are  designed  to  help  us.  Medicine,  psychological  treatment  for  phobias, 
and  simulations  that  hone  one’s  skills  are  indeed  useful,  if  not  extremely  important.  However,  minimizing  the 
detrimental  or  unwanted  aspects  should  be  considered  before  releasing  an  application  of  technology  just  for  the  sake  of  the 
technology. 

We  reiterate,  “If  the  medium...were  a membrane  separating  us  from  the  object...we  should  be 
relative  to  it  in  the  same  condition  as  we  are  to  air  or  water  in  which  we  are  immersed 
(Aristotle,  350  BC).” 
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Preface: 


"Introduction  to  Computing  (CmpsclOl)"  is  an  undergraduate  introductory  course  that  enrolls  roughly  1700 
students  per  year  from  widely  diverse  nontechnical  fields.  For  most  students,  "Intro,  to  Computing'*  is  their  only 
exposure  to  the  Computer  Science  discipline.  The  students  are  taught  to  design,  code,  and  test  what  are  for  an 
introductory  course,  fairly  sophisticated  programs,  which  are  structured,  modular,  and  thoroughly  documented.  This 
course  meets  most  of  the  requirements  for  an  introductory  Computer  Science  course,  as  described  in  the  American 
Computing  Machinery  "ACM  ’78"  paper.  "Intro,  to  Computing"  is  one  of  many  undergraduate  courses  that  deal  with 
quantitative  subject  matter  at  an  introductory  level  and  that  students  are  required  to  take  as  a general  education 
requirement.  Due  to  large  class  sizes,  instructors  of  such  courses  often  have  difficulty  involving  students  actively  in  the 
learning  process  during  class  time,  let  alone  adapting  course  content  to  students’  personal  interests  and  goals  in  order  to 
make  learning  more  meaningful.  Students,  for  the  most  part,  retain  the  subject  matter  long  enough  to  pass  the  final  exam 
which  results  in  inert,  unprocessed,  and  therefore  unusable  knowledge.  CBI  (computer-based  instruction)  can  play  a key 
role  in  solving  such  problems  by  contextualizing  the  content  of  instruction  to  the  most  relevant  and  current  interests  of 
each  student.  By  providing  interestingly  relevant  contexts,  and  learner  choice  of  context,  the  locus  of  control  is  placed 
with  the  student  and  not  the  instructor.  This  instructional  strategy  enables  students  to  relate  content  to  personal 
experiences  and  preferences  and  therefore  construct  personally  meaningful  knowledge  that  is  usable  and  transferable  to 
new  situations  in  the  future. 


Introduction: 

Most  college  courses  can  be  described  as  having  an  instructional  core  with  supplemental  features.  For  example, 
in  one  course  the  core  might  be  the  lectures,  with  text(s)  considered  supplemental  and  useful  only  to  the  extent  that  they 
elaborate  on  the  materials  covered  fully  in  lectures.  In  another  course,  labs  might  be  the  core  while  lectures  and  texts 
would  be  peripheral.  Introductory  undergraduate  courses  are  no  different.  Large  college  institutions  refer  to  these  courses 
as  "service  courses"  due  to  the  fact  that  students  enrolled  in  them  are  non-majors  to  the  discipline  being  taught,  but  are 
required  to  take  them  as  foundational  to  their  major,  or  as  a general  education  base.  Service  courses  are  generally  taught  in 
lecture  halls  that  seat  anywhere  from  50  to  600  (or  more)  students  and  the  subject  matter  covered  is  usually  domain 
general  knowledge.  Instructors  typically  use  projection  devices  or  movable  blackboards  as  teaching  aids,  and  spend  most 
of  the  period  lecturing  from  a platform  through  a microphone.  Some  of  those  courses  offer  small  group  "recitation 
classes”  to  help  the  students  in  understanding  or  applying  the  subject  matter.  This  is  where  the  "examples"  and 
"applications"  of  the  content  are  explained  or  demonstrated. 

The  format  and  teaching  techniques  described  above  have  one  thing  in  common:  courses  are  organized  in  linear 
order  from  the  instructor's  point  of  view  and  therefore  every  student  receives  the  same  instruction  at  the  same  pace  using 
the  same  context.  One  instructional  certainty  is  that  the  way  in  which  an  expert  organizes  a body  of  knowledge  is  not  the 
same  as  the  way  that  a novice  does  (Glaser,  1985).  Another  instructional  certainty  is  that  what  might  be  a relevant  and 
meaningful  context  for  one  student  is  not  necessarily  relevant  and  meaningful  to  another.  The  list  goes  on.  All  to  say... 
and  this  is  hardly  revolutionary...  that  the  focus  must  be  on  the  student’s  learning  and  especially  on  the  diversity  of  ways 
in  which  students  learn.  However  structuring  a course  around  students’  diverse  learning  styles  and  individual  differences 
becomes  very  problematic  in  a large  class.  The  purpose  of  this  dissertation  study  is  to  show  how  computer  technology 
can  help  in  overcoming  such  constraints  by  adapting  the  content  of  "service  courses"  to  interestingly  relevant  contexts 
and  allowing  the  student  to  choose  the  most  personally  relevant  context  at  the  time  of  instruction.  While  this  adaptive 
strategy  does  not  cover  many  of  the  individual  differences  that  impact  learning,  it  follows  a learner  control  format  of 
providing  learner  choice  of  context,  which  can  be  modified  or  extended  to  provide  learner  choice  of  lesson  sequence, 
pacing,  content  structure,  and  other  instructional  variables  that  are  considered  adaptive  to  cognitive  styles  and  prior 
knowledge.  This  approach  gives  a sense  of  ownership  of  content  to  the  learner  and  promotes  relevancy. 

Theoretical  Framework: 

Robert  Yager  (Yager,  1989)  provides  a rationale  for  using  " personal  relevance " (defined  by  Eisner  (Eisner,  1985) 
as  an  orientation  for  the  science  curriculum  which  emphasizes  the  primacy  of  personal  meaning  for  students  by 
developing  programs  that  focus  on  their  interests  and  personal  experiences)  as  the  primary  focus  and  organizer  for 
planning  a science  curriculum  in  schools.  He  argues  that  personal  meaning  and  understanding  precedes  ", social  adaptation " 
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(defined  by  Eisner  (1985)  as  an  orientation  for  curriculum  planning  that  focuses  on  societal  concerns)  of  individuals 
making  other  orientations  (Eisner,  1985)  defines  five  basic  orientations  for  the  school  curriculum:  personal  relevance, 
social  adaptation  and  reconstuction,  academic  rationalism,  technology,  and  development  of  cognitive  processes)  for 
curriculum  planning  in  science  inappropriate  due  to  their  inability  to  promote  a useful  understanding  of  the  discipline  and 
a positive  attitude  or  desire  among  students  to  study  more  science.  He  goes  on  to  argue  that  typical  school  programs  that 
are  planned  by  curriculum  developers,  textbook  companies,  and  individual  teachers,  do  not  seem  to  take  into  consideration 
the  student,  his/her  interests,  motivations,  and  personal  experiences.  Yager  believes  that  a personal  relevance  approach  to 
designing  a science  curriculum  would  resolve  such  issues  as  well  as  past  failures  of  science  education.  Eisner  (Eisner, 

1985)  supports  this  argument  by  stating  that  students  must  have  some  investment  in  the  learning  experience  for  it  to 
become  educational.  Students  must  actively  participate  and  share  in  curriculum  development  in  order  to  make  real  choices 
available  otherwise  "schooling  is  likely  to  be  little  more  than  a series  of  meaningless  routines,  tasks  undertaken 
to  please  someone  else's  conception  of  what  is  important"  (Yager,  1989).  Eisner  likens  the  personal  relevance  orientation 
to  "open  schools"  (lack  of  domain  structure)  and  to  programs  organized  around  student  interests. 

If  one  were  to  espouse  the  above  approach,  then  structuring  a science  curriculum,  or  any  curriculum,  around 
students'  interests  and  personal  experiences  is  not  an  easy  task.  The  unique  and  individual  interpretation  of  educational 
tasks  is  largely  dependent  on  prior  knowledge  and  the  construction  and  reconstruction  of  the  activities  that  students 
engage  in  for  experimental  purposes  (Renninger,  1992).  Individual  interest  as  defined  by  Renninger  includes  both  the 
stored  knowledge  and  the  particular  relation  (value)  of  the  individual  with  the  task  relative  to  other  activities  that  he  or 
she  engages  in.  Since  interest  is  expected  to  vary  between  individuals  and  therefore  influence  the  way  in  which  they  act,  a 
strategy  is  needed  to  identify  individual  interest  and  match  the  educational  task  to  the  particular  interest  in  order  to 
increase  personal  relevance. 

Whereas  Renninger  operationalizes  the  definition  of  interest  by  emphasizing  the  contributions  that  individual 
differences  bring  to  the  understanding  of  an  educational  task  (knowledge/value  system),  Hidi  (Hidi  & McLaren,  1990) 
takes  a different  perspective  on  how  interest  affects  learning  or  cognitive  performance  and  that  is  through  the 
interestingness  of  the  context  or  what  has  been  termed  by  Krapp  (Krapp,  1989a)  as  situational  interest.  The  focus  of 
situational  interest  is  on  the  environment  and  the  features  or  characteristics  of  the  stimuli  present  in  the  environment, 
verses  the  individual  and  his  or  her  personal  knowledge  and  values.  Interestingly,  the  interaction  between  personally 
generated  interest  (individual  interest)  and  environmentally  generated  interest  (situational  interest)  is  crucial  to  both 
perspectives  (Hidi,  1990;  Hidi  & Baird,  1986).  A particular  form  of  situational  interest  is  known  as  text-based  interest. 
Interest  is  elicited  by  using  interesting  analogies  and  examples  in  certain  text  segments,  and  through  ideas,  topics,  and 
themes  that  are  relevant  to  the  educational  goal.  Research  results  on  text-based  interest  can  be  described  as  increasing 
motivation  and  comprehension  but  not  enhancing  overall  learning  unless  coupled  with  other  organizers  of  text  such  as 
structural  features  (Hidi  & McLaren,  1990). 

Key  Argument: 

Building  on  both  perspectives  of  interest,  situational  verses  individual,  the  researcher  proposes  that  the 
interaction  between  situational  interest  and  individual  interest  is  a crucial  factor  in  promoting  increased  motivation,  and 
effective  knowledge  acquisition  and  transfer.  Individual  interest  in  an  educational  task  or  activity  will  motivate  the  student 
to  engage  in  the  learning  process,  and  situational  interest  will  provide  the  authenticity  and  real  world  significance  of  the 
task  to  provide  a meaningful  experience  and  personal  relevance.  The  kev  factor  here  is  adapting  the  educational  task, 
situationallv-  to  students'  individual  interests.  The  question  is  then  how  do  educators  choose  educational  contexts  that  are 
relevant  to  students'  individual  interests  and  adapt  the  educational  task  situationally  to  those  contexts? 

Let's  start  by  operationalizing  the  concept  of  individual  or  personal  interests.  Personal  interests  lie  on  a 
continuum  ranging  from  internally  generated  interests  to  externally  generated  interests,  with  internally  generated  interests 
beginning  with  personal  beliefs,  values,  and  concerns,  and  extending  to  the  surrounding  environments  and  beyond  (see 
figure  A). 


Figure  A 

See  Appendix  A of  the  1996  AECT  Proceedings 


Personal  Interests  Continuum 


Individual  interests  can  be  driven  by  one  or  many  of  the  personal,  social,  or  environmental  structures  provided  in 
the  above  continuum  but  the  affordances  (optimal  opportunities)  that  each  of  these  structures  provide,  and  the  unique 
interaction  of  an  individual  with  these  affordances,  will  dictate  the  most  highly-valued  interest  at  a particular  point  in 
time. 

There  is  also  a certain  degree  of  overlap  among  internally  generated  and  externally  generated  interests  and  a ripple 
effect  is  created  if  one  takes  a circular  view  of  the  continuum.  To  illustrate,  Lewin  (Lewin,  1935)  defines  the  life  space  of 
an  individual  as  a mental  sphere  that  consists  of  the  person  and  the  psychological  environment  that  exists  for  that  person 
at  any  given  time.  Life  space  is  the  extension  of  the  individual  from  the  personal  dimension  to  the  social  and 
environmental  dimensions  and  is  dynamically  influenced  by  the  needs  within  each  dimension  and  the  interpersonal 
interactions  among  those  needs.  Personal  interests  therefore,  change,  based  on  the  movement  of  the  individual  in  the  life 
space  (see  figure  B). 


Figure  B 

See  Appendix  A of  the  1996  AECT  Proceedings 

Lewin's  Visualization  of  the 
Life  Space  of  an  Individual 

Contexts  for  educational  content  can  be  created  to  match  the  personal,  social,  and  environmental  dimensions  of 
the  life  space  of  an  individual  and  students  can  freely  select  the  most  personally  relevant  context  depending  on  their 
location  on  the  "personal  interests"  continuum  and  the  foreseeable  affordances  that  a particular  context  might  bring. 

Creating  Educational  Contexts  Based  on  Interest: 

Herndon  (Herndon,  1987)  investigated  learner  interests  as  a variable  that  can  enhance  motivation  in  learners  and 
incorporated  students'  interests  in  an  instructional  unit  on  how  to  solve  conditional  syllogisms  (a  form  of  reasoning  in 
which  from  two  given  or  assumed  propositions  called  the  premises  & having  a common  or  middle  term  a third  is 
deduced).  He  collected  data  from  an  interest  survey  given  to  all  the  students  that  participated  in  the  study  and  built  the 
interests  inventory  using  the  most  frequently  reported  learner  interests.  He  used  this  inventory  to  structure  the  examples 
of  the  'interests'  version  of  the  instructional  unit  and  found  that  significantly  more  subjects  in  this  group  were  willing  to 
return  to  the  task  of  solving  syllogisms  than  the  subjects  in  the  'no  interests'  version  of  the  instructional  unit.  The 
interest  survey  was  based  on  the  work  of  Sarbin  (Sarbin,  1964)  who  described  human  experience  as  an  interaction  of  five 
'ecologies’  or  facets  of  the  environment  within  which  the  individual  'forms  collections  of  highly  valued  interests'.  The 
ecologies  or  environmental  spheres  according  to  Sarbin  are:  Self-Maintenance  (objects  and  activities),  Spatial  (places), 
Social  (persons).  Normative  (knowledge  and  skills),  and  Transcendental  (ideas  and  beliefs).  Five  questions  were  asked  in 
the  survey  that  were  derived  from  four  ecologies.  The  questions  asked  for  three  favorite  possessions,  three  things  one 
likes  to  do  best,  three  places  that  one  likes  best,  three  highly  valued  persons,  and  three  things  in  which  one  has  a valued 
knowledge  or  skill. 

Selecting  contexts  for  educational  content  can  take  on  a similar  process  by  conducting  a survey  on  a subset  of 
the  experimental  population  that  asks  students  to  prioritize  knowledge  domains  such  as  science,  social  studies,  and/or 
humanities,  from  most  interesting  to  least  interesting  based  on  their  desirability  as  an  educational  context  for 
introductory,  quantitative  subject  matter.  The  domains  will  represent  the  personal,  social,  and  environmental  dimensions 
of  the  life  space  of  an  individual  using  the  structures  of  the  "personal  interests"  continuum  defined  above.  For  example, 
sociology  might  be  a context  that  represents  the  social  dimension  and  incorporates  issues  dealing  with  the  family  and  the 
community.  Daily  life  might  be  a context  that  represents  the  personal  dimension  and  incorporates  issues  such  as  personal 
beliefs  and  well  being.  Fiction  might  be  a context  that  represents  the  environmental  dimension  and  incorporates 
transcendental  or  supernatural  issues.  An  inventory  of  contexts  can  be  included  in  the  survey  and  contexts  with  high 
ratings  will  be  identified  as  most  relevant  to  students*  interests.  Educational  tasks  can  then  be  contextualized  to  all  or  part 
of  the  selected  contexts  depending  on  the  need  and  applicability  of  the  particular  course  content. 
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Let’s  suppose  that  six  educational  contexts  have  been  identified  as  highly  relevant  to  students'  interests:  science, 
daily  life,  education,  business,  sociology,  and  the  arts.  Computer-based  instruction  can  present  this  inventory  of 
contexts  to  students  enrolled  in  an  introductory  course  and  students  will  be  asked  to  select  their  most  preferred  context  for 
learning  the  subject  matter  of  that  course.  The  content  can  then  be  contextualized  to  the  most  preferred  context.  This 
process  can  be  repeated  each  time  the  student  interacts  with  the  instruction.  Individual  interest  will  be  promoted  through 
choice  of  context  and  situational  interest  through  contextualizing  of  content.  The  resulting  interaction  will  increase  the 
valence  (perceived  challenges  or  possibilities)  of  the  task  and  the  student  can  then  take  advantage  of  such  a learning 
strategy  to  construct  meaningful  knowledge  and  as  a result  transfer  the  knowledge  to  applicable  and  new  situations  in  the 
future. 

The  interaction  or  contextual  effect  resulting  from  the  above  instructional  strategy  becomes  more  crucial  when 
the  educational  task  or  content  has  a negative  valence  for  a particular  individual.  The  assumption  here  is  that  Quantitative 
subject  matter  generally  has  a negative  valence  for  non-technical  students  and  a transformation  in  valence  from  negative 
to  positive  is  what’s  being  sought.  This  transformation  can  be  realized  by  embedding  the  educational  task  in  a context 
that  becomes  "personally  relevant". 

Contextualizing  Instruction: 

Preparing  alternative  context  lessons  and  matching  them  to  students'  interests  is  a time-consuming  task  (Ross, 
McCormick,  Krisak  & Anand,  1985)  that  teachers  are  not  able  to  perform  particularly  in  undergraduate  education  due  to 
large  class  sizes.  One  of  the  most  powerful  features  of  computers  is  their  ability  to  adapt  instruction  to  learner 
characteristics  by  individualizing  lessons  on  predetermined  instructional  variables.  It  is  important  to  distinguish  here 
between  adapting  and  individualizing  instructional  lessons.  In  the  literature  of  instructional  technology,  individualizing 
instruction  implies  incorporating  decisions  about  pacing,  sequencing,  and  selection  of  content  which  is  commonly 
known  as  "learner  control"  or  "internal  control”  (Hannafin  & Ross,  1984)  stated  that  learner  control  or  internal 
control",  is  adaptive  if  students  have  the  knowledge  and  motivation  to  make  correct  decisions  about  their  learning  needs). 
Programmed  instruction  is  an  example  of  individualizing  instruction  that  is  based  on  "program  or  external  control"  (Ross 
& Morrison,  1988).  The  l iamer  is  branched  to  the  appropriate  lesson  segment  depending  on  the  evaluation  of  the  learner's 
performance.  Several  criticisms  of  program-controlled  instruction  have  been  levied  by  critics  including  Kannafm 
(Hannafin,  1984),  and  Ross  & Morrison  (Ross  & Morrison,  1988),  concerning  the  accuracy  of  the  designer’s  judgement 
on  when  and  how  to  direct  the  branching  process,  and  the  reliability  and  validity  of  the  evaluation  criteria. 

AHaptinp  instruction  is  adjusting  the  educational  task  or  activity  to  students’  prior  knowledge  and  needs  or 
relating  the  instruction  to  students'  goals  and  interests.  It  can  be  achieved  through  various  instructional  strategies  one  of 
which  is  adapting  the  content  of  the  instruction  to  learners’  interests  and  preferences  (different  situations)  known  as 
contextualizing  instruction.  It  is  anchoring  the  lesson  in  an  authentic  context  that  promotes  personal  relevance.  If  the 
learner  perceives  that  significant  personal  needs  are  being  met  by  the  instruction  then. relevance  will  be  increased  (Keller 
& Suzuki,  1988),  and  the  learner  will  be  more  motivated  to  engage  in  the  learning  task.  This  would  facilitate  the 
assimilation  and  integration  of  new  knowledge  by  associating  the  task  with  familiar  and  personally  meaningful 
events  (Ross,  1983). 

Another  instructional  strategy  that  claims  to  be  adaptive  and  flourished  with  the  onset  of  Computer  Assisted 
Instruction,  (CAI),  is  personalizing  instruction.  Personalizing  instruction  is  incorporating  personal  information  about  the 
student  such  as  birth  date,  names  of  familiar  people,  hobbies,  and  other  specific  events  into  the  instructional  lesson. 
Personalization  of  instructional  material  in  most  commercial  software  is  a piecemeal  approach  to  adapting  the  context  to 
students'  individual  interests  and  can  create  a false  facade  of  personal  relevance  which  may  result  in  shallow  processing  of 
the  content  rather  than  conceptual  understanding  and  transfer  of  knowledge.  Personalizing  is  adaptive  only  to  the  extent  of 
providing  personal  relevance  through  recognizing  individual  characteristics  and  interests  and  not  through  CQntCktVfllizittfi 
the  task  to  individual  interests. 

The  above  adaptive  strategies  are  necessary  but  not  sufficient  to  create  the  appropriate  balance  between  individual 
interest  and  situational  interest.  Contextualizing  instruction  in  this  study  goes  a step  further  by  adapting  the  content  to  a 
personal,  social,  or  environmental  context  that  has  real-world  significance  and  an  applied  focus  or  goal.  It  anchors  the 
abstract  concepts  of  the  educational  task  in  a situation  that  has  a theme  and  a goal.  For  example,  let’s  assume  that  the 
most  preferred  context  for  learning  statistics  for  a particular  student  is  pop  music.  Instead  of  presenting  the  statistical 
concept  of  MODE  as  the  score  which  has  the  highest  frequency  in  a distribution,  it  would  be  presented  as  the  number  of 
weeks  that  Michael  Jackson  maintained  his  single  hit  at  number  one  in  the  charts  (presuming  that  he  had  the  highest 
frequency  amongst  a group  of  artists  during  a specified  time  period).  Other  related  content  would  be  presented  using  the 
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same  theme.  A goal  would  be  to  compute  the  percentile  ranks  of  the  artists’  hits  as  they  move  up  and  down  the  charts. 
Embedding  the  content  in  a personally  relevant  theme  and  giving  it  an  applied  goal  minimizes  the  problem  of 
“disconnected  knowledge”  ai.d  maximizes  the  opportunity  of  learning  "knowledge  as  information  to  be  applied  or  used" 
(Perkins,  1992). 

Foundational  Research: 

Several  studies  were  done  by  Ross  et.  al.  (Anand  & Ross,  1987;  Ross,  1983;  Ross,  McCormick  & Krisak, 

1986;  Ross  et  al.,  1985)  to  determine  the  effects  of  adapting  the  context  of  quantitative  material  to  student  interests  and 
background  as  well  as  personalizing  arithmetic  material  to  elementary  school  children  using  computer-assisted 
instruction.  Adapting  instruction  in  the  above  research  studies  consisted  of  adjusting  or  matching  the  examples  of  the 
subject  matter  to  identifiable  student  variables  or  needs  such  as  prior  knowledge,  academic  major,  and  individual  interests. 

In  one  experiment,  preservice  teachers  were  given  a lesson  on  probability  using  three  context  treatments: 
abstract  or  non-adaptive,  an  education  context  and  a medical  context.  The  preservice  teachers  matched  with  the  education 
context  performed  better  than  the  abstract  and  medical  contexts  on  immediate  and  delayed  near  transfer  problems  (context 
related  problems),  and  better  than  the  abstract  gioup  on  far  transfer  problems  (novel  situations).  Another  experiment  was 
conducted  to  validate  the  adaptive  context  strategy  to  students  background  by  providing  nursing  students  with  the  same 
material.  The  results  were  equally  effective.  The  group  of  nursing  students  that  was  matched  with  the  medical  context 
performed  better  on  all  learning  outcomes.  The  bottom  line  was  the  relevance  of  the  context  to  the  subjects’  interests. 

Ross  et.  al.  (Ross  et  al.,  1985)  furthered  the  research  on  adaptive  instruction  by  testing  whether  individual-based  context 
selection  (learner-adaptive  group)  was  more  advantageous  than  group-based  context  selection  (standard-adaptive  group).  In 
the  individual -based  approach  students  were  asked  to  rank-order  (from  most  to  least  favored)  four  areas  (education,  medical, 
sports,  and  abstract)  that  were  used  as  contexts  for  presenting  word  problems.  Students  were  either  matched  with  their 
most  favored  choice  during  instruction  (learner-choice  adaptive  strategy)  or  their  least  favored  choice  (learner-choice 
nonadaptive).  The  group-based  approach  used  a standard-adaptive  strategy  by  matching  an  entire  group  of  subjects  to  the 
context  that  represented  their  academic  major  (using  two  separate  experiments,  nursing  majors  were  assigned  the  medical 
context  and  education  majors  were  assigned  the  education  context).  The  hypothesis  was  that  a medical  context  would  be 
more  favorable  to  nursing  majors  and  an  education  context  more  favorable  to  education  majors  and  consequently  more 
meaningful  learning  would  occur.  A standard-nonadaptive  strategy  was  administered  as  a control  in  which  the  assigned 
group(s)  received  the  word  problems  in  an  abstract  context  (the  word  problems  did  not  reflect  any  real-world  application  of 
the  mathematical  concept  under  study).  The  results  favored  the  adaptive  treatments  on  a variety  of  achievement  and 
attitude  measures  with  particular  positive  effects  on  transfer  measures  in  one  adaptive  group.  No  differences  were  reported 
between  the  individualized  and  the  group  adaptive  strategies.  The  authors  indicated  that  the  homogeneity  of  each  group 
and  differences  in  prior  knowledge  may  have  contributed  to  those  results.  The  authors  also  pointed  out  that  the  contexts 
for  the  learner-choice  adaptive  and  standard  adaptive  groups  were  almost  identical  because  nearly  all  the  subjects  in  the 
learner-choice  adaptive  groups  selected  the  same  context  as  the  standard  adaptive  groups.  In  a previous  study,  Ross  et.  al. 
(Ross  et  al.,  1985),  reported  similar  findings  and  attributed  the  no-difference  on  achievement  measures  between  learner- 
choice  and  standard  adaptation  strategies  to  yet  another  factor  and  that  is  the  limitation  of  context  options. 

The  above  research  findings  clearly  indicate  the  benefits  of  adapting  context  to  students'  interests  but  they  do  not 
clarify  whether  the  benefits  were  due  to  the  personal  relevance  of  the  context  to  the  learner.  The  contexts  provided  in  the 
above  experiments  were  based  on  students’  academic  major  and  not  on  knowledge  domains  that  could  represent  personal, 
social,  or  environmental  dimensions  of  the  life  space  of  an  individual.  Individual-based  selection  and  adaptation  of  most 
preferred  context  was  not  favored  as  a unique  strategy  but  the  overall  posttest  mean  difference  between  learner-choice 
adaptive  and  learner-choice  nonadaptive  was  18.8  (78.3  - 59.5)  versus  8.5  (69.4  - 60.9)  for  the  group-based  adaptive  and 
non-adaptive  strategies  (Ross  et  al.,  1986). 

A contrast  in  the  application  of  context  to  instruction  was  investigated  in  a series  of  studies  by  Bransford  & 
Johnson  (Bransford  & Johnson,  1972).  The  studies  provided  evidence  on  using  prior  knowledge  as  a tool  to  aid 
comprehension.  The  major  assumption  was  that  "contextual  information  is  necessary  for  the  ongoing  process  of 
comprehension"  and  that  prior  knowledge  must  be  semantically  encoded  (related  to  prior  or  familiar  knowledge)  to 
become  usable  and  accessible  in  new  situations.  The  experiments  involved  the  interpretation  of  the  underlying  structures 
of  abstract  sentences  and  the  treatments  were  varied  using  a Context  Before  reading  the  passage.  Context  After  reading  the 
passage,  and  No  Context  Before  or  After  reading  the  passage.  The  context  was  provided  using  an  appropriate  picture. 
Other  experiments  used  a topic  for  the  passage  to  activate  a suitable  context.  Results  showed  that  using  the  Context 
Before  (providing  the  context  before  the  passage),  greatly  improved  comprehension  and  recall  scores.  Bransford  and 
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Johnson  suggest  that  contextual  cues  must  be  present  at  input  in  order  to  aid  comprehension.  The  data  also  provided 
evidence  that  students  who  were  not  provided  with  the  context  (picture  or  topic  in  this  case)  prior  to  reading  the  passage 
were  actively  searching  for  a situation  to  anchor  the  passage  in.  The  topic  and  the  picture  helped  the  students  activate 
semantic  contexts  of  prior  knowledge  by  functioning  as  a mnemonic  device  (Lachman  et.  al.  (Lachman  & Pompi,  1967; 
Lachman  & Dooling,  1971)  suggest  that  knowledge  of  the  topic  facilitates  retention  by  functioning  as  a mnemonic 
device.  Bransford  and  Johnson  (Bransford  & Johnson,  1972)  however  view  the  role  of  the  topic  as  something  more  than 
generating  associations  and  that  is  to  aid  comprehension  through  creating  meaningful  contexts  of  abstract  information). 

The  above  experiments  provided  context  as  an  advance  organizer  to  help  relate  incoming  information  to  prior 
knowledge  in  a meaningful  manner.  Depending  on  the  type  of  advance  organizer,  there  could  be  several  interpretations  n 
the  content.  Even  though  the  key  issue  was  the  timing  of  presenting  the  context  rather  than  the  choice  of  context,  the 
effects  of  contextual  content  on  knowledge  comprehension  were  verified. 

Renninger  (Renninger,  1992)  conducted  a study  on  interest  and  mathematical  word  problem  solving  using  a 
population  of  fifth  and  sixth  graders  where  a total  of  12  word  problems,  3 with  interesting  contexts  and  3 with 
noninteresting  contexts  at  the  student's  mastery  level,  and  3 with  contexts  that  were  identified  interests  and  3 with 
contexts  that  were  identified  noninterests  at  the  student's  instructional  level.  In  other  words,  the  mastery  level  word 
problems  were  based  on  situational  interest  and  the  instructional  level  word  problems  were  based  on  individual  interest. 
The  dependent  variables  were  accuracy  and  type  of  error  committed.  Findings  revealed  that  students  were  more  accurate  in 
completion  of  mastery  than  instructional  problems.  Individual  interest  and  gender  had  a joint  effect  on  the  type  of  error 
committed.  For  example,  boys  made  more  error  in  setting  up  noninteresting  problems  (presumably  because  interest  is 
more  likely  to  assist  in  understanding  these  problems)  and  girls  made  more  errors  in  setting  up  interesting  problems 
(presumably  because  interest  is  more  likely  to  be  a distraction  than  an  aid  on  these  tasks).  On  a similar  interest  and 
reading  study,  students  had  an  easier  time  engaging  in  (or  being  distracted  by)  contexts  of  interest  than  contexts  of 
noninterest.  Renninger's  research  seems  to  suggest  that  individual  interest  generates  task  engagement  based  on  the 
challenges  that  students  set  and  the  kinds  of  questions  with  which  they  feel  most  familiar  with,  whereas  situational 
interest  is  more  spontaneous  and  can  cause  interest  by  surprise  and  therefore  produce  unexpected  results  in  certain 
situations.  Further  analysis  suggests  that  individual  interest  tends  to  have  long-lasting  effects  on  a person's  knowledge 
and  values  while  situational  interest  may  have  only  a short  term  effect  and  marginally  influence  an  individual's  knowledge 
and  values  (Hidi,  1990;  Hidi  & McLaren,  1990).  Renninger's  research  strengthens  the  argument  that  each  type  of  interest 
has  its  own  drawbacks  and  that's  more  of  a reason  why  the  interaction  between  situational  interest  and  individual  interest 
is  crucial. 

Purpose  of  Study. 

Capitalizing  on  the  significance  of  the  effects  of  contextual  information  on  knowledge  acquisition  and  transfer, 
and  the  varied  effects  of  individual  interest  and  situational  interest  on  promoting  task  engagement,  the  purpose  of  this 
dissertation  study  is  to  'tease  out'  the  confounding  factors  that  dampened  the  effectiveness  of  individualized  (learner-choice) 
adaptive  strategies  in  the  Ross  et.  al.  (Ross  et  al.,  1986)  study  by  eliminating  group-based  standard  adaptive  approach, 
expanding  the  interests  inventory  of  contexts  based  on  identified  students  interests,  anchoring  the  content  of  quantitative 
subject  matter  in  the  identified  contexts,  and  allowing  individual  choice  of  context  at  the  time  of  instruction.  This  study 
attempts  to  show  that  providing  interesting  contextualized  choices  will  have  a causal  effect  on  personal  relevance  and 
knowledge  acquisition. 

Research  Question: 

Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student's  most  interesting  context,  increase 
relevance,  and  facilitate  knowledge  acquisition  and  transfer,  versus  adapting  to  a student's  least  interesting  context  or  non- 
adapting? 

Subordinate  Questions: 

1 . Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student’s  most  interesting  context 
increase  relevance  of  content? 

2.  Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student’s  most  interesting  context 
improve  attitude  toward  the  content? 

3.  Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student's  most  interesting  context 
improve  achievement  on  recall  measures? 
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4.  Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student's  most  interesting  context 
improve  achievement' on  application  or  near  transfer  measures? 

5.  Will  adapting  CBI  of  introductory  undergraduate  course  content  to  a student's  most  interesting  context 
improve  achievement  on  inference  or  far  transfer  measures? 

6.  Does  the  most  interesting  context  for  a particular  individual  vary  from  one  lesson  to  another? 

7.  Is  there  an  interaction  between  the  context  of  the  lesson  and  the  context  of  the  achievement  measure? 

8.  Is  there  an  interaction  between  the  context  of  the  lesson  and  the  level  of  learning  outcome  being  measured? 

9.  Is  there  a relationship  between  the  student's  academic  major  and  his/her  choice  of  context? 

Methods  & Procedures: 

A pilot  study  was  conducted  to  test  the  main  research  question  above  and  some  of  the  subordinate  questions. 

This  section  will  describe  the  design,  the  methodology  and  the  results  of  the  pilot  study. 

Pilot  Study: 

Design: 

The  design  of  the  pilot  study  was  a one  factor  quasi-experimental  design  with  a pre-post  measure  on  the 
dependent  variable  relevance  and  a posttest-only  measure  on  the  dependent  variable  knowledge  acquisition  and  transfer. 

Subjects: 

The  subjects  were  a heterogeneous  group  of  undergraduate  students  enrolled  in  computer  science  100,  an 
introductory  "service  course"  that  familiarizes  students  with  general  computer  applications  (word  processing,  data  base 
management,  and  spreadsheets).  The  subjects  participated  in  this  study  as  part  of  completing  class  assignments  but  were 
provided  with  an  alternative  assignment  if  they  did  not  wish  to  participate. 

Medium: 

The  instructional  lessons  were  developed  using  authoring  tools  to  produce  computer-based  instructional 
modules.  The  content,  which  was  elementary  statistics,  was  contextualized  to  eight  domains:  sociology,  the  arts,  daily 
life,  business,  science,  education,  music,  and  fiction  (see  sample  below).  There  was  also  an  abstract  lesson.  Each  context 
was  composed  of  one  lesson  which  contained  3 modules:  central  tendency  & skewness,  variability,  and  z-scores  and  the 
normal  distribution.  The  treatments  were  identical  in  size,  length,  screen  layout,  and  content  sequence  and  complexity. 

Method: 

140  subjects  were  randomly  assigned  to  3 treatment  groups:  learner  most  preferred  adaptive:  subjects  were 
administered  the  most  preferred  choice  of  context,  learner  least  preferred  adaptive:  subjects  were  administered  the  least 
preferred  choice  of  context,  and  non-adaptive  or  abstract  group:  subjects  were  administered  the  abstract  lesson  (no 
context).  Subjects  in  all  three  groups  were  asked  to  select  their  most  preferred  context  based  on  "the  desirability  of  the 
domains  as  contexts  for  statistics  lessons".  Then  they  were  asked  to  rate  the  remaining  domains  on  a scale  of  1 to  5 from 
most  interesting  to  least  interesting.  There  was  a brief  explanation  of  each  context  to  facilitate  the  selection  and  rating 
process.  An  attitude/motivation  survey  was  administered  prior  to  and  after  the  treatments  to  assess  any  change  in  students' 
attitudes/motivation  towards  statistics.  The  survey  was  an  adaptation  of  two  highly  reliable  attitudinal  surveys:  Keller's 
IMMS  (Instructional  Materials  Motivation  Survey)  which  had  reliability  estimates  of  0.96  using  a Cronbach  alpha 
measure,  and  Roberts  (Roberts  & Bilderback,  1980)  SAS  (Statistics  Attitude  Survey)  which  had  reliability  estimates  of 
0.90  using  the  Spearman-Brown  formula.  The  questions  selected  from  both  surveys  were  aimed  at  measuring  students 
attitudes  towards  statistics  and  the  perceived  relevance  of  statistics  to  students'  interests  and  experiences.  Included  in  the 
pre-treatment  survey  were  demographic  questions  asking  students  about  their  background  (major,  profession,  age,  and 
number  of  prior  statistics  courses),  and  general  questions  assessing  students'  degree  of  familiarity  with  the  statistical 
concepts  to  be  taught  in  the  instructional  modules.  An  abstract,  or  no-context  achievement  posttest,  testing  recall, 
application,  synthesis,  and  evaluation  of  the  statistical  concepts  learned  in  the  CBI  lessons  was  administered  at  the  end  of 
all  3 treatments. 

Results: 

14  students  dropped  the  course  prior  to  conducting  the  study.  12  students  did  not  participate  in  the  study.  1 14 
students  completed  the  pre-attitude  survey.  81  students  out  of  the  1 14  completed  all  parts  of  the  study.  The  28%  attrition 
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rate  was  largely  attributed  to  the  fact  that  the  instructor  of  the  course  had  given  the  students  the  option  of  dropping  one  of 
their  assignments  for  the  semester  and  participation  in  the  study  was  considered  as  one  assignment.  The  researcher 
believes  that  the  motive  for  participation  became  almost  voluntary  due  to  the  criteria  set  by  the  instructor. 

Out  of  the  8 1 complete  records  compiled  from  the  study,  only  28%  selected  their  most  preferred  context  in  direct 
relationship  to  their  major.  This  supports  the  principle  that  individual  interest  changes  and  that  students'  interests  are  not 
necessarily  related  to  their  academic  major.  In  other  descriptive  statistical  analysis,  the  study  showed  that  28%  of  the 
subjects  (28  seems  to  be  the  magic  number  here)  selected  'daily  life'  as  their  most  preferred  domain,  followed  by  24%  for 
'sociology',  14%  science,  12%  music,  1 1%  fiction,  10%  business,  1%  education,  and  0%  selected  'the  arts'. 

No  significant  differences  were  reported  on  the  dependent  variable  "relevance  of  statistics"  and/or  "attitude 
towards  statistics"  when  using  a single  factor  ANOVA  with  the  independent  factor  being  gain  scores  reported  from  the 
pre-post  attitudinal  survey.  Further  analysis  is  required  to  test  whether  there  were  any  specific  changes  on  selected 
components  of  the  attitudinal  scale  such  as  professional  relevance,  personal  relevance,  educational  relevance  or  general 
confidence  in  one's  abilities  towards  the  subject  matter.  The  Cronbach  alpha  measure  on  the  attitude  instrument  was 
0.920. 

The  p-value  reported  on  the  dependent  variable  achievement,  was  0.10  using  a 95%  confidence  interval  with  the 
means  being:  12.17  for  the  abstract  (control)  or  no-context  group,  11.42  for  the  least  preferred  adaptive,  and  10.19  for  the 
most  preferred  adaptive.  The  variance  for  each  of  the  levels  was:  13.01  for  the  abstract  (control)  group,  15.37  for  the 
least-preferred  adaptive  group,  and  8.29  for  the  most-preferred  adaptive  group.  Further  analysis  on  the  posttest  items 
showed  a p-value  of  0.05  on  application  questions,  and  a p-value  of  0.09  on  the  synthesis  and  evaluation  questions 
combined.  No  significance  was  reported  on  the  recall  questions.  The  estimated  reliability  measure  on  the  achievement 
posttest  was  0.672  using  a KR20  index.  The  overall  posttest  mean  was  1 1 .26%  and  the  distribution  of  the  test  scores 
was  clearly  normal.  A correlation  of  0.40  was  reported  between  the  pretest  measure  of  'the  degree  of  familiarity  or 
knowledge  of  basic  statistical  concepts'  and  the  achievement  posttest  scores. 

Discussion: 

The  results  of  the  pilot  study  suggest  that  it  may  be  more  useful  to  look  at  interactions  rather  than 
main  effects.  The  variations  obtained  on  the  different  learning  outcomes  on  the  achievement  scores  (p-value  of  0.05  on 
application  measures,  0.09  on  evaluation/synthesis  measures  and  0.6  on  recall  measures)  indicate  that  the  independent 
variable  may  have  greater  effects  on  interactions  between  groups  and  types  of  learning  outcomes.  The  high  within  group 
variance  suggest  that  individual  differences  in  aptitude,  prior  knowledge,  and  learning  styles  may  have  played  a major  role 
in  masking  a significant  main  effect.  The  experimental  population  was  truly  heterogeneous.  In  the  Ross  et.  al  studies 
(Ross  et.  al,  1986  #14)  the  groups  were  homogeneous  in  that  the  individuals  in  a particular  treatment  group  had  the  same 
major  and  were  at  the  same  college  level.  A larger  n is  therefore  required  to  detect  significance  in  a heterogeneous 
population.  Finally,  a closer  look  at  the  CB1  treatments  revealed  that  they  had  minimal  user  interactivity.  Emphasizing 
context  as  the  primary  instructional  strategy,  the  treatments  did  not  incorporate  any  other  instructional  strategy  that 
promoted  "mental  effort"  (Salomon,  1983a)  defines  the  construct  of  "mental  effort"  as  "the  number  of  non-automatic 
elaborations  applied  to  a unit  of  material",  p.  42)  and  engagement.  The  treatments  could  best  be  described  as 
"contextualized  tutorials"  or  page  turners. 

Based  on  the  above  findings,  the  following  design  improvements  will  be  implemented  in  the  actual  dissertation 
study: 

• due  to  the  extremely  high  variance  of  the  three  groups,  a larger  sample  of  subjects  (at  least  twice  the  size  of 
the  pilot  sample)  v/ill  be  used  to  be  able  to  detect  better  significance; 

• to  justify  that  attrition  is  consistent  and  congruent  among  the  three  treatment  groups,  random  assignment  of 
subjects  will  be  done  at  the  time  of  CBI  instruction  by  randomizing  the  treatments  instead  of  the  subjects; 

• to  improve  the  internal  validity  of  the  study,  the  subjects  will  be  selected  from  an  introductory  statistics  class 
(Statistics  200),  and  the  CBI  will  be  given  as  an  independent  study  module  that  will  count  as  homework  credit; 

• two  contexts  will  be  eliminated  due  to  their  low  ratings:  'the  arts',  and  'education'; 

• the  remaining  contexts  (sociology,  science,  daily  life,  business,  music,  and  fiction)  will  be  carefully 
examined  in  term.''  of  their  personal,  social,  and  environmental  implications  and  will  be  restructured  to  form 
distinctly  relevant  and  non-relevant  contexts  for  the  experimental  population; 

• the  achievement  posttest  will  include  questions  from  each  of  the  context  areas  to  find  out  whether  subjects 
will  do  better  on  test  items  that  match  their  lesson  context; 
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• to  enhance  mental  effort  and  task  engagement,  an  inquisitory  dialogue  will  be  used  when  presenting  the 
stimulus  (instead  of  the  story-telling  approach  that  seemed  to  be  prevalent  in  the  pilot  study  treatments),  and 
practice  questions  with  corrective  feedback  will  also  be  incorporated; 

® since  significance  was  better  approached  when  analyzing  the  scores  on  the  posttest  based  on  subscales  of 
learning  outcomes,  the  posttest  questions  will  be  better  delineated  to  be  able  to  measure  three  distinct  learning 
outcomes:  recall,  application,  and  inference. 

Implications  on  Teaching  and  Learning: 

Many  schools  are  faced  with  the  problem  of  delivering  consistent,  high  quality,  low  cost  introductory  courses. 

An  entire  course  (including  lectures,  text,  computer  guided  instruction,  item  pool  for  exam-generation  and  a handbook  for 
teaching  assistants)  which  could  be  administered  at  relatively  low  cost,  should  prove  highly  marketable. 

To  maximize  the  efficiency  of  information  transfer  and  to  encourage  students  to  view  service  courses  as  an  added 
skill  rather  than  a drudgery,  the  instructional  design  strategy  of  "adapting  instruction  to  students'  most  personally  relevant 
context"  described  above  can  be  implemented  to  deliver  a high  quality  course  to  an  increasingly  heterogeneous  mix  of 
students. 

Other  instructional  variables  such  as  structural  features  of  the  content,  lesson  sequencing  and  pacing,  and  type  of 
feedback  can  also  be  manipulated  using  this  adaptive  strategy  to  support  a more  student-centered  approach.  The  researcher 
maintains  though  that  contextualizing  content  of  quantitative  subject  matter  remains  the  primary  variable  in  promoting 
meaningful  learning.  Further  investigation  of  this  instructional  strategy  through  an  enhanced  study  is  currently  underway. 

Sample  of  Contextualized  Instruction 

Introductory  screen  of  a statistical  lesson  contextualized  to  the  music  domain: 

You  are  working  for  the  WKRZ  radio  station , and  your  mission  is  to  do  a market  analysis  to  find  out  what  type  of  music 
should  WKRZ  play ; and  at  what  time  of  the  day , in  order  to  attract  the  largest  share  of  college  students . You  want  to  be 
able  to  tell  potential  advertisers  that  WKRZ  has  a high  volume  of  young  listeners  as  a target  population.  You  conduct  a 
survey  on  a random  sample  of  college  students  and  you  include  questions  such  as: 

1.  What  type  of  music  do  you  listen  to?  (classic  rock , rock , new  wave , etc.) 

2.  What  time  of  day  do  you  mostly  listen  to  the  radio  and  to  what  station? 

3.  How  many  hours  per  week  do  you  listen  to  music  in  general? 

4.  What  proportion  of  weekly  hours  do  you  spend  listening  to  the  radio? 

5.  How  much  money  do  you  spend  per  semester  on  CD's?  tapes? 

Introductory  screen  of  the  same  content  contextualized  to  the  business  domain: 

You  are  working  for  an  investment  company  (Wall  Street  One)  and  your  mission  is  to  do  a market  analysis  to  find  out 
what  type  of  investment  packages  should  Wall  Street  One  set  up , in  order  to  target  middle  income  families  in  the  area. 
You  want  to  be  able  to  compete  with  already  existing  and  established  investment  packages  by  offering  a leading  edge  that 
is  specific  to  your  target  population.  You  conduct  a survey  on  a random  sample  of  middle  income  families  in  the  area  and 
you  include  questions  such  as: 

1.  How  much  money  do  you  spend  on  investments  such  as  stocks,  bonds,  IRA  accounts,  etc.,  per  week? 

2.  How  many  hours  per  month  do  you  spend  managing  your  investments? 

3.  How  much  money  do  you  spend  per  week  paying  off  credit  cards  and  loans? 

4.  What  time  of  day  do  you  mostly  talk  to  your  investment  agent  or  brokerage  firm  concerning  questions  and 
developments  of  your  investment  accounts? 

Introductory  screen  of  the  abstract  or  no  contexi  lesson: 

You  have  conducted  a survey  on  a random  sample  of  a target  population  and  you  want  to  describe  the  data  resulting  from 
the  survey  using  some  basic  statistical  measures  that  are  typical  of  the  values  in  the  data  set. 
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You  cannot  possibly  list  all  the  values  you  collected  because  there  are  too  many.  So  you  want  to  make  sure  that  your 
analysis  is  indicative  of  the  trends  of  the  target  population  that  you  surveyed. 

You  asked  many  questions  in  your  survey  such  as: 

1 . The  number  of  hours  they  spend  doing  some  activity  per  week. 

2.  The  time  of  day  they  spend  doing  that  activity. 

3.  The  cost  of  doing  the  activity  in  dollars  per  week. 

4.  The  proportion  of  time  they  spend  on  some  activity  verses  other  activities. 
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Abstract 


The  purpose  of  this  study  was  to  investigate  the  effects  of  using  individual,  cooperative,  and  collaborative 
learning  structures  with  a computer  simulation.  College  students  implemented  one  of  the  three  learning  structures  while 
using  a computer  simulation  designed  to  teach  the  technical  and  procedural  aspects  of  accounting.  Results  indicated  that 
performance  scores  were  high  regardless  of  learning  structure.  However,  students  who  worked  alone  expressed 
significantly  more  continuing  motivation  for  their  learning  structure  than  students  who  worked  with  a partner. 

Responses  to  student  interviews  revealed  somewhat  mixed  feelings  for  the  small  group  structures.  Furthermore, 
observation  of  the  small  groups  indicated  that  students  in  the  cooperative  dyads  exhibited  significantly  more  discussion 
and  provided  more  answers  to  their  partners'  questions  than  students  in  the  collaborative  dyads.  Implications  for 
implementing  small  group  structures  with  computer-based  instruction  are  provided. 

Implementing  Individual  and  Small  Group 
Learning  Structures  with  a Computer  Simulation 

There  has  been  a great  deal  of  interest  recently  in  implementing  small  group  learning  structures  in  a variety  of 
educational  settings.  The  success  of  small  group  methods  in  classrooms  (cf.  Johnson  & Johnson,  1989;  Sharan,  1980; 
Slavin,  1990)  has  prompted  educational  technologists  to  examine  the  effect  of  employing  small  group  strategies  with 
computer-based  instruction  (CBI).  According  to  Johnson,  Johnson  & Stanne  (1985),  small  group  structures  reduce  the 
social  isolation  inherent  in  the  design  of  most  CBI  by  providing  students  the  opportunity  to  discuss  what  they  are 
learning. 


A number  of  research  studies  have  been  conducted  to  examine  the  effect  of  implementing  small  group  methods 
with  CBI.  Some  researchers  have  found  that  student  achievement  and  attitude  increases  when  small  group  strategies  are 
used  with  CBI  (Dalton,  Hannafin,  & Hooper,  1989;  Hooper,  Temiyakam,  & Williams,  1993;  Johnson,  Johnson,  & 
Stanne,  1985;  Schlecter,  1990).  However,  others  have  reported  that  individual  and  small  group  structures  are  equally 
effective  when  employed  with  computer  instruction  (Carrier  & Sales,  1987;  Orr  & Davidson,  1993). 

The  mixed  results  for  using  small  groups  with  CBI  might  be  due  to  the  type  of  small  group  method 
implemented  in  these  studies.  Two  of  the  most  frequently  used  small  group  methods  are  cooperative  and  collaborative 
learning.  While  both  share  similar  elements,  Smith  and  MacGregor  (1992)  have  indicated  that  these  methods  can  be 
placed  irto  distinct  categories  of  learning  approaches  that  require  joint  intellectual  effort.  These  categories  are  based  on 
the  struc  .re  required  for  each  method. 

Cooperative  learning  is  the  most  highly  structured  of  all  small  group  methods  (Smith  & MacGregor,  1992)  and 
has  been  defined  as  students  working  together  on  tasks  that  require  interdependent  goals  and  rewards  (Smith,  1989). 
Cooperative  learning  activities  are  structured  to  include  positive  interdependence,  face-to-face  interaction,  interpersonal  and 
group  skills,  and  individual  accountability  (Johnson  & Johnson,  1988).  Furthermore,  group  rewards  are  often  provided  to 
promote  cooperation  among  students  (Slavin,  1991). 

Small  group  methods  such  as  peer  collaboration  require  less  structure  than  cooperative  learning  (Davidson, 

1985,  Smith  & MacGregor,  1992).  Damon  and  Phelps  (1989)  have  indicated  that  the  two  elements  of  equality  and 
mutuality  distinguish  collaboration  from  cooperation.  Equality  is  achieved  when  students  with  similar  abilities  work  to 
help  each  other  learn.  Mutuality  is  influenced  by  the  structure  of  a task  when  groups  are  required  to  work  together. 
Division  of  labor  structures,  found  in  many  cooperative  settings,  are  low  in  mutuality  since  each  student  can  work  alone 
without  exchanging  information.  Peer  collaboration  establishes  a more  mutual  status  than  cooperative  learning  (Damon 
& Phelps,  1989). 

The  purpose  of  the  current  study  was  to  investigate  the  effects  of  implementing  cooperative,  collaborative,  and 
individual  learning  structures  with  a computer  simulation  in  accounting.  The  Accounting  Education  Change 
Commission  has  recommended  the  addition  of  several  instructional  methods  to  accounting  curricula,  including  active 
student  participation,  small  group  work,  and  the  creative  use  of  technology  (ACC.;  1990).  The  accounting  simulation 
used  in  this  study  was  designed  for  an  ACC.  grant  awarded  to  the  School  of  Accountancy  in  1991 . The  course  designers 
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and  faculty  wanted  to  determine  if  small  group  learning  structures  recommended  by  the  ACC.  (1990)  were  as  effective  as 
the  individual  learning  structure  originally  implemented  with  the  computer  simulation. 

There  is  very  little  research  on  using  small  group  learning  methods  or  computer-based  technology  in  accounting 
education  A recent  study  by  Ravenscroft,  Buckless,  McCombs,  Sc  Zuckerman  (1993)  indicated  that  college  students  in 
an  introductory  accounting  course  who  worked  in  teams  achieved  higher  grades  than  students  who  worked  individually. 
However,  no  differences  were  found  for  students  enrolled  in  a course  on  auditing.  Furthermore,  Oglesbee,  Bitner,  and 
Wright  (1988)  reported  that  students  enrolled  in  an  introductory  accounting  course  who  used  CBI  performed  better  than 
those  in  a lecture  format.  However,  those  results  disappeared  once  the  researchers  controlled  for  prior  achievement. 

Method 

Subjects 


Subjects  for  the  study  were  105  college  students  (63  females,  42  males)  enrolled  in  an  accounting  course  at  a 
large  southwestern  university.  All  subjects  were  completing  prerequisite  requirements  for  entry  into  an  upper  division, 
professional  program  in  the  College  of  Business.  Students  had  completed  a first  semester  course  in  introductory 
accounting  with  a grade  of  B or  better.  They  were  concurrently  enrolled  in  a second  semester  introductory  accounting 
course  and  in  a computer  laboratory  course.  The  study  was  conducted  in  this  computer  lab. 

Materials 

The  materials  used  in  this  study  were  a computer  simulation  designed  to  teach  students  the  technical  and 
procedural  aspects  of  accounting  and  a set  of  student  booklets  designed  to  facilitate  the  implementation  of  the  simulation 
in  either  an  individual,  cooperative  or  collaborative  environment. 

Computer  simulation.  The  computer  simulation,  entitled  An  Inttodilgtion  to  Accounting:  A Business 
Simulation  (Birney  & Smith,  1994),  was  designed  using  the  framework  of  an  accounting  firm.  All  treatment  groups 
used  the  computer  simulation  without  any  variations  in  the  program.  Ii  provided  pre-instruction,  information,  practice, 
and  feedback  on  the  topic  of  how  to  prepare  adjusting  journal  entries. 

In  the  simulation,  each  student  was  an  employee  of  an  accounting  firm  and  reported  to  his  or  her  office  to  begin 
each  computer  session.  Once  in  the  office,  the  employees  "desk"  contained  many  tools  such  as  a pop-up  calculator,  a 
FAX  machine  for  messages  from  clients,  computer  E-mail  for  messages  from  the  boss,  and  a calendar.  The  laboratory 
assignments  were  listed  on  the  employee  s calendar,  which  directed  a student  to  the  tasks  to  be  performed  and  the  order  of 
completion  for  the  tasks.  Tasks  that  were  completed  showed  a red  check  mark  next  to  them. 

The  first  part  of  the  simulation  was  a tutorial  session  providing  information  and  examples  about  how  to  prepare 
adjusting  journal  entries.  Content  and  example  screens  were  followed  by  28  practice  items  v/hich  included  fill  in  the 
blanks,  numeric  calculations,  true  or  false  statements,  and  constructed  response  (journal  entry  preparation  and  1 -account 
analysis).  The  exercises  required  a student  to  use  and  apply  various  skills  and  knowledge  in  order  to  complete  the  next 
two  parts  of  the  simulation  designated  as  practice  sets. 

Both  practice  sets  were  presented  through  a business  simulation.  At  the  beginning  of  each  simulation,  a pop-up 
calendar  appeared  on  the  screen,  showing  a list  of  the  clients1  books  that  needed  to  be  adjusted  (Qucng  Heating  Co.  and 
Holt  Laundry).  Both  practice  sets  required  a student  to  make  decisions  about  the  need  for  adjustments  to  the  clients' 
books,  to  prepare  accrual  or  deferral  adjustments,  to  observe  the  posting  of  the  adjustments,  and  to  prepare  an  adjusted 
trial  balance. 

The  format  design  of  both  practice  sets  included  four  steps.  First,  a memo  from  the  client  was  shown  on  the 
screen  listing  various  transactions  that  had  occurred  for  the  client’s  business.  The.  instructions  asked  the  student  to  select 
the  transactions  which  would  require  adjustments.  Next,  the  computer  simulation  displayed  various  imernai  and  external 
documents  pertaining  to  the  transactions  which  the  student  reviewed  in  the  memo.  The  instructions  asked  the  student  to 
prepare  the  necessary  adjusting  journal  entry  for  each  source  document.  The  third  step  instructed  the  student  to  post  the 
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adjustments  to  the  general  ledger.  The  fourth  step  showed  the  student  the  various  posted  accounts  from  the  general  ledger 
and  instructed  them  to  prepare  an  adjusted  trial  balance. 

A practice  exam  followed  the  second  practice  set.  This  exam  included  21  items  which  were  similar  in  content 
and  design  structure  to  the  tutorial  and  simulation  practice  exercises.  The  practice  exam  was  scored  by  the  computer  and 
constituted  10%  of  a student's  grade.  After  the  practice  exam,  the  computer  provided  a review  of  questions  that  the 
student  answered  incorrectly.  This  review  provided  both  the  right  answer  and  an  explanation  of  why  that  answer  was 
correct. 


Student  booklets.  Three  different  student  booklets  were  designed  to  provide  directions  and  procedures  to 
implement  an  individual,  cooperative,  or  collaborative  learning  structure.  The  basic  design  for  each  booklet  was  a 
content  and  activity  outline  of  the  computer  module  for  Preparing  Adjusting  Entries.  In  addition,  different  prompts  were 
printed  on  the  booklets  to  help  students  consistently  utilize  treatment-specific  instructions. 

The  booklet  for  the  individual  treatment  group  included  a set  of  instructions  describing  the  procedures  each 
student  was  to  follow  in  the  computer  lab.  It  directed  students  to  work  individually  on  each  part  of  the  simulation.  The 
booklet  included  generic  prompts  in  the  introduction  such  as  "use  the  space  provided  in  your  booklet  to  note  any 
questions  you  have  about  the  material."  It  also  supplied  procedural  prompts  throughout  the  booklet  such  as  "you  are 
now  ready  to  start  working  on  Practice  Set  I." 

The  booklet  for  the  cooperative  treatment  group  included  a set  of  instructions  describing  the  procedures  each 
student  would  follow  in  a cooperative  dyad.  The  introduction  explained  that  students  would  take  turns  performing  the 
role  of  preparer  and  checker.  The  duties  of  each  role  were  described  in  the  introduction;  students  were  informed  that  the 
preparer  should  "do  the  work  at  the  keyboard"  and  the  checker  should  "check  the  work  and  provide  assistance  as  needed." 
The  booklet  included  procedural  prompts  throughout  to  remind  to  frequently  rotate  their  roles  and  to  ensure  that  each 
member  of  the  dyad  would  practice  the  skills  and  knowledge  included  in  the  simulation.  The  booklet  also  reminded 
students  to  shift  the  mouse  and  keyboard  over  to  the  preparer  each  time  there  was  a switch  in  roles. 

The  booklet  for  the  collaborative  treatment  group  included  a set  of  instructions  describing  the  procedures  each 
student  would  follow  in  a collaborative  dyad.  The  introduction  provided  general  directions  such  as  "work  by  yourself  at 
your  own  computer  and  make  notes  of  questions  you  want  to  discuss  with  your  partner."  The  booklets  also  included 
procedural  prompts  throughout  which  directed  students  to  work  alone  during  designated  portions  of  the  lesson,  make 
notes  of  questions  to  ask  their  partner,  and  discuss  these  questions  with  their  partner  at  specified  times  during  simulation. 

Procedures 

This  study  was  implemented  over  a two  week  period  in  a computer  laboratory  as  part  of  required  course 
activities.  Prior  to  the  study,  students  were  informed  that  they  would  participate  :n  a research  study  and  that  extra  credit 
would  be  awarded  to  students  who  attended  each  of  the  four,  75  minute  class  sessions. 

Several  weeks  before  the  study,  seven  intact  classes  were  randomly  assigned  to  either  the  individual,  cooperative, 
or  collaborative  treatment  conditions.  Before  assignment  to  treatments  was  made,  test  scores  from  the  first  course 
examination  were  collected  and  analysed  to  established  equality  between  the  classes.  A one-way  analysis  of  variance  of 
these  scores  revealed  no  significant  difference  between  classes  for  prior  achievement.  The  following  section  describes  the 
procedures  followed  by  subjects  in  each  of  the  treatments. 

Individual  treatment.  Students  working  individually  followed  the  standard  lab  format  for  the  class.  On  Day  1, 
each  individual  received  a student  booklet  and  were  told  to  note  where  they  stopped  at  the  end  of  each  lab  session. 

Students  then  worked  alone  on  the  computer  simulation.  On  Day  2,  individuals  received  their  booklets  as  they  entered 
the  lab  aud  were  instructed  to  proceed  with  the  simulation.  On  Day  3,  individuals  were  told  that  they  should  finish  the 
simulation  by  the  end  of  the  day.  Students  completed  the  practice  exam  and  review  after  practice  set  2.  On  Day  4, 
students  completed  a pencil  and  paper  posttest  and  then  entered  their  answers  on  the  computer.  Immediately  following 
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the  posttest,  students  completed  an  attitude  survey.  Twenty-eight  students  from  the  individual  treatment  participated  in 
an  interview  about  the  study  after  they  completed  the  posttest  and  attitude  survey. 

Cooperative  treatment.  Students  in  this  treatment  condition  used  a cooperative  learning  structure  suggested  by 
Kagan  (1989).  This  strategy  required  two  students  to  work  together  to  complete  the  simulation,  alternating  the  roles  of 
preparer  and  checker.  Positive  interdependence  was  achieved  through  the  structured  tasks  each  partner  had  to  perform,  and 
by  providing  the  mutual  goal  of  reaching  consensus  on  a solution.  In  addition,  the  pair  received  the  same  grade  on  the 
practice  exam,  which  constituted  10%  of  their  total  unit  grade. 

On  Day  1,  students  were  randomly  assigned  to  a dyad;  each  dyad  was  assigned  to  a computer.  Students  were 
informed  that  they  would  receive  the  same  score  for  the  practice  exercises  and  practice  exam.  Each  student  was  given  a 
booklet  and  told  to  note  where  they  stopped  at  the  end  of  each  lab  session.  Students  then  worked  cooperatively  on  the 
computer  simulation.  On  Day  2,  each  student  received  their  booklet  and  were  reminded  to  switch  roles  as  prompted.  On 
Day  3,  dyads  were  again  reminded  to  switch  roles  and  were  told  that  they  should  finish  the  simulation  by  the  end  of  the 
day.  Dyads  completed  the  practice  exam  and  review  together  after  practice  set  2.  During  the  practice  exam,  students  were 
preparers  for  half  of  the  questions  and  checkers  for  half  of  the  question.  On  Day  4,  each  student  individually  completed  a 
pencil  and  paper  posttest  and  then  entered  their  answers  on  the  computer.  Immediately  following  the  posttest,  each 
student  completed  an  attitude  survey.  Twenty-nine  students  from  the  cooperative  treatment  participated  in  an  interview 
after  they  completed  the  posttest  and  attitude  survey. 

Collaborative  treatment.  Students  in  this  treatment  condition  used  a collaborative  learning  strategy  while 
completing  the  simulation.  Each  dyad  member  individually  worked  through  one  part  of  the  simulation  at  a time  and  were 
directed  to  note  questions  or  concepts  they  wanted  to  discuss  with  their  partner.  Students  were  prompted  at  specified 
points  in  the  lesson  to  consult  with  their  partner  to  discuss  these  questions.  Positive  interdependence  was  achieved  by 
structuring  activities  so  that  students  would  benefit  from  sharing  their  ideas  and  responding  to  other’s  questions  and 
comments. 

On  Day  1 , students  were  randomly  assigned  as  dyads  to  computers  that  were  adjacent  to  each  other.  Each  student 
was  given  a booklet  and  told  to  note  where  they  stopped  at  the  end  of  each  lab  session.  Students  then  worked 
collaboratively  on  the  computer  simulation.  On  Day  2,  each  student  received  their  booklet  and  were  reminded  to 
individually  complete  part  of  each  practice  set  and  then  consult  using  the  notes  they  made  in  the  booklet.  On  Day  3, 
students  were  again  reminded  to  use  the  collaborative  method  ana  were  told  that  they  should  finish  the  simulation  by  the 
end  of  the  day.  Students  individually  completed  the  practice  exarr.  f<  nd  review  after  practice  set  2,  and  then  collaborated 
with  their  partner  on  notes  they  made  during  the  practice  exam.  Each  student  received  an  individual  grade  on  the  practice 
exam.  On  Day  4,  each  student  individually  completed  a pencil  and  paper  posttest  and  then  entered  their  answers  on  the 
computer.  Immediately  following  the  posttest,  each  student  completed  an  attitude  survey.  Thirty  students  from  the 
collaborative  treatment  participated  in  an  interview  about  the  study. 

Criterion  Measures 

Criterion  measures  for  this  study  were  student  achievement,  attitude,  time  on  task,  student  interactions,  and 
responses  to  student  interviews. 

Achievement  was  measured  using  a 35  item,  paper  and  pencil  posttest  designed  to  assess  student  mastery  ot  the 
skills  taught  in  the  computer  simulation.  The  test  items  were  of  a similar  nature  to  the  tutorial  and  practice  set  items 
and  asked  for  knowledge  answers,  numeric  calculations,  true  or  false  choices,  and  constructed  responses.  There  were  21 
selected  response  items  and  14  constructed  response  items  on  the  posttest.  The  KR-21  reliability  of  this  test  was  .75. 

Attitude  was  measured  using  an  eight  item,  paper  and  pencil  survey.  Students  used  a five  point,  likert-type  scale 
(1  ~ strongly  agree,  5 = strongly  disagree)  to  record  their  r : ponses  to  the  following  items: 

1 . The  unit  was  interesting. 

2.  I learned  a lot  from  this  unit. 

3.  I did  well  on  the  posttest. 
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4.  I like  working  alone/with  a partner. 

5.  I learn  more  working  alone/with  a partner. 

6.  I would  take  other  courses  structured  like  this  one. 

7.  The  grading  for  this  unit  was  fair. 

8.  I am  very  comfortable  using  computers. 

The  attitude  survey  was  administered  immediately  after  the  posttest.  The  Cronbach  alpha  reliability  of  this  survey  was 
.76. 


Time  on  task  was  tracked  by  the  computer  simulation  which  captured  the  amount  of  time  students  spent  on  the 
tutorial,  both  practice  sets,  and  the  review. 

The  number  of  student  interactions  exhibited  by  dyads  in  the  cooperative  and  collaborative  learning  structures 
were  observed  and  recorded  on  an  observation  sheet.  This  observation  sheet  included  interaction  behaviors  that  other 
researchers  have  suggested  as  necessary  for  successful  group  work  (Klein  & Pridemore,  1994;  Webb,  1982,  1987).  These 
interaction  behaviors  were  grouped  into  five  categories  of  questioning  (asking  a question),  answering  (answering  a 
question),  encouraging  (giving  praise  or  unsolicited  help),  discussing  (talking  about  content  or  task),  and  off-task  (verbal 
and  non  verbal  behaviors).  Trained  observers  were  stationed  among  four  dyads  to  observe  each  dyad  and  record 
observations  at  approximately  two  minute  intervals  during  the  first,  second,  and  third  lab  periods.  The  observers  watched 
each  dyad  for  five  minute  intervals  during  the  fourth  lab  period.  During  a pilot  study,  three  observers  watched  the  same 
dyad  for  several  two  minute  intervals  and  recorded  their  behaviors.  Reliability  was  based  on  all  three  observers  having 
similar  totals  for  this  dyad.  The  inter-rater  reliability  between  observers  was  .85. 

Student  interviews  were  conducted  using  a five  question,  written  interview  protocol.  Questions  about  the 
computer  lesson  and  learning  structures  were  based  on  student  responses  obtained  from  interviews  conducted  in  the  pilot 
study.  Students  volunteered  to  participate  in  these  one-to-one  interviews.  They  were  conducted  by  one  interviewer  after 
each  student  completed  the  posttest  and  attitude  survey.  Approximately  84%  of  the  students  who  participated  in  the  study 
(87  out  of  105)  volunteered  to  be  interviewed. 

Design  and  PataAnglysis, 

This  study  used  a posttest  only,  control  group  design  with  the  one  independent  variable  of  learning  structure 
(individual,  cooperative  and  collaborative).  The  dependent  variables  included  student  achievement,  attitudes,  time  on  task, 
student  interactions,  and  responses  to  interviews. 

A one-way  analysis  of  variance  (ANOVA)  was  conducted  on  the  posttest  data.  ANOVA  was  also  conducted  on 
data  for  overall  time  on  task  and  time  spent  on  each  section  of  the  simulation.  Separate  3 (learning  structure)  X 5 (item 
selection)  chi-square  analyses  were  conducted  for  each  item  on  the  attitude  survey.  The  number  of  interactions  exhibited 
by  students  in  the  cooperative  and  collaborative  treatments  were  totalled  and  categorized  as  questioning,  answering, 
encouraging,  discussing,  and  off-task.  Separate  chi-square  analyses  were  conducted  on  each  category  of  interaction 
behaviors.  Finally,  responses  to  student  interviews  were  summarized  and  reported  as  percentages. 

Results 

Student  Achievement 

The  mean  posttest  achievement  score  was  3 1 .30  for  students  in  the  individual  learning  structure,  3 1 . 1 9 for 
students  in  the  collaborative  learning  structure,  and  30.94  for  students  in  the  cooperative  learning  structure.  Student 
performance  across  all  groups  was  3 1 . 14  or  89%.  ANOVA  indicated  that  the  difference  between  the  treatment  groups  on 
the  posttest  was  not  statistically  significant. 
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Student  Attitudes 

Student  responses  to  the  attitude  items  revealed  that  most  students  thought  that  the  unit  was  interesting  (76  out 
of  105'  M = 2.12)  and  believed  that  they  learned  a lot  from  it  (86  out  of  105;  M = 1 .98).  In  addition,  most  students 
thought  that  they  did  well  on  the  posttest  (71  out  of  105,  M = 2.13),  that  the  grading  for  the  unit  was  fair  (88  out  of 
105,  M = 1.97),  and  that  they  were  very  comfortable  using  computers  (85  out  of  105,  M = 1.86). 

Separate  3 (learning  structure)  X 5 (item  choice)  chi-square  analyses  were  conducted  for  each  item  on  the  attitude 
survey.  These  analyses  indicated  a significant  difference  between  the  learning  structures  on  one  of  the  eight  altitude 
items.  Chi-square  indicated  a significant  difference  for  the  item  ”1  would  take  other  courses  that  were  structured  the  same 
as  this  one,"  X 2 = (8,  N = 105)  = 14.49,  p < .05.  Data  revealed  that  70%  of  the  students  in  the  individual  learning 
structure  agreed  or  strongly  agreed  with  this  item,  compared  to  only  36%  of  the  students  in  the  cooperative  structure  and 
22%  in  the  collaborative  structure.  In  contrast,  only  12%  of  the  students  in  the  individual  learning  structure  disagreed  or 
strongly  disagreed  with  this  item,  compared  to  42%  of  the  students  in  the  cooperative  structure  and  28%  of  the  students 
in  the  collaborative  structure. 


Time  data  revealed  that  the  average  number  of  minutes  spent  on  the  entire  simulation  was  140.18  for  students 
who  worked  alone,  145.44  for  students  who  worked  cooperatively,  and  144.15  for  students  who  worked  collaboratively. 
ANOVA  indicated  that  the  differences  between  these  means  was  not  statistically  significant. 

In  addition  to  overall  time  spent  on  the  simulation,  data  were  collected  for  the  amount  of  time  students  spent  on 
the  tutorial,  both  practice  sets,  and  the  review.  These  data  indicated  a significant  difference  between  treatments  for  the 
review  portion  of  the  simulation,  E(2,102)  = 3.40,  p < .05.  The  average  number  of  minutes  spent  on  review  was  5.31 
for  students  who  worked  alone,  1 1.35  for  students  who  worked  cooperatively  and  7.25  for  students  who  worked 
collaboratively.  Post  hoc  analysis  using  Tukey  HSD  pairwise  comparisons  revealed  that  students  who  worked 
cooperatively  spent  significantly  more  time  on  review  than  students  who  worked  alone,  (p  < .05). 

Student  Interactions 

Table  1 summarizes  the  number  of  student  interactions  exhibited  by  dyads  in  the  cooperative  and  collaborative 
learning  structures.  These  interaction  behaviors  were  grouped  into  five  categories  of  questioning,  answering, 
encouraging,  discussing,  and  off-task  behaviors.  Chi-square  analyses  were  performed  on  each  of  the  five  different 
behaviors  to  determine  the  influence  of  learning  structure.  These  analyses  indicated  a significant  difference  between 
cooperative  and  collaborative  learning  structures  on  two  of  the  five  behaviors.  Students  in  cooperative  dyads  exhibited  a 
total  of  459  discussion  behaviors  while  those  in  the  collaborative  dyads  exhibited  218  discussion  behaviors,  X2  = (1,  N = 
36)  = 12.29,  p < .001 . Students  in  cooperative  dyads  also  provided  significantly  more  responses  to  their  partners' 
questions  (194)  than  students  in  the  collaborative  dyads  (136),  X2  = (1,  N = 36)  = 10.19,  p < .01. 

Responses  to  Student  Interviews 

Table  2 provides  a summary  of  student  responses  to  the  five  interview  questions.  These  data  suggest  that  the 
majority  of  students  responded  that  the  simulated  practice  sets  were  the  most  helpful  feature  of  the  computer  lesson. 
Seventy-nine  percent  of  individual  subjects,  48%  of  cooperative  subjects,  and  57%  of  collaborative  subjects  listed  the 
practice  as  the  most  helpful  aspect  of  the  lesson. 

Participants  were  also  asked  what  aspects  of  the  student  booklets  were  helpful.  Thirty-six  percent  of  individual 
subjects,  55%  of  cooperative  subjects,  and  43%  of  collaborative  subjects  listed  the  organization  and  instructions  as  most 
the  most  helpful  feature  of  the  booklets.  However,  57%  of  individual  subjects,  45%  of  cooperative  subjects,  and  43%  of 
collaborative  subjects  indicated  that  they  did  not  use  the  booklets  throughout  the  entire  lesson. 


Time  on  Task 


Responses  to  questions  about  learning  structure  revealed  that  57%  of  individual  subjects,  52%  of  cooperative 
subjects,  and  43%  of  collaborative  subjects  indicated  a preference  for  working  alone  over  working  with  a partner. 

Finally,  71%  of  individual  subjects,  62%  of  cooperative  subjects,  and  60%  of  collaborative  subjects  responded  that 
working  with  a partner  would  be  more  enjoyable  than  working  alone.  However,  only  46%  of  individual  subjects,  38%  of 
cooperative  subjects,  and  40%  of  collaborative  subjects  said  they  would  learn  more  working  with  a partner. 

Discussion 

The  purpose  of  the  current  study  was  to  investigate  the  effects  of  using  cooperative,  collaborative,  and  individual 
learning  structures  with  a computer  simulation.  College  students  enrolled  in  an  accounting  course  implemented  one  of 
the  three  learning  structures  while  using  a computer  simulation  to  learn  how  to  prepare  adjusting  entries  to  close  clients' 
books. 


Results  indicated  that  scores  on  the  performance  test  were  high  regardless  of  learning  structure.  The  average 
achievement  score  was  89%  for  subjects  in  all  three  treatments.  This  result  is  likely  due  to  the  students  who  participated 
in  the  study.  Prior  to  this  study,  all  subjects  had  completed  an  introductory  accounting  course  with  a grade  of  B or  better; 
most  indicated  that  they  would  pursue  a college  major  in  accounting.  Furthermore,  scores  from  a test  given  prior  to  the 
study  suggested  that  subjects  had  a high  degree  of  ability  to  perform  tasks  similar  to  those  taught  in  the  computer 
simulation.  It  is  likely  that  these  students  would  perform  well  regardless  of  their  placement  in  an  individual  or  small 
group  learning  structure. 

This  explanation  is  consistent  with  results  found  by  other  researchers.  Ravenscroft  et  al.(1993)  found  that  small 
group  learning  increased  scores  of  students  in  an  introductory  accounting  course,  but  did  not  influence  performance  in  an 
advanced  course  on  auditing.  Oglesbee,  Bitner,  and  Wright  (1988)  reported  that  prior  achievement  had  a strong  influence 
on  scores  in  an  accounting  course  regardless  of  the  medium  used  to  deliver  instruction. 

Results  for  time  on  task  indicated  that  students  spent  the  same  amount  of  overall  time  working  on  the 
simulation  regardless  of  learning  structure.  In  addition,  no  differences  were  found  for  time  spent  on  the  tutorial  and  on 
both  practice  sets.  However,  cooperative  subjects  did  spend  significantly  more  time  on  the  review  portion  of  the  lesson 
than  subjects  in  the  other  learning  structures.  One  possible  explanation  for  this  finding  is  that  students  in  the 
cooperative  learning  structure  felt  that  the  benefits  of  review  would  increase  because  of  additional  discussion  and 
elaboration  of  the  content. 

The  result  that  small  group  subjects  did  not  spend  more  time  than  individual  subjects  on  the  simulation  is  not 
consistent  with  other  cooperative  learning  studies.  In  fact,  Slavin  (1990)  indicated,  "most  studies  that  ha  measured 
time  on  task  have  found  higher  proportion  of  engaged  time  for  cooperative  learning  students  than  for  control  students" 
(p.47). 


Observations  of  students  in  the  individual  learning  structure  may  shed  some  light  on  why  these  students  spent  as 
much  time  on  the  simulation  as  those  in  the  small  group  structures.  These  observations  suggested  that  individuals 
frequently  accessed  the  on-line  reference  book,  which  was  bright  red  in  color  and  noticeably  visible  on  screens  throughout 
the  laboratory.  This  likely  increased  time  on  task  for  individuals. 

While  results  suggested  that  the  three  learning  structures  were  equally  effective,  findings  from  the  attitude  survey 
revealed  that  students  who  worked  alone  expressed  significantly  more  continuing  motivation  for  their  learning  structure 
than  students  who  used  the  small  group  structures.  Approximately  70%  of  students  in  the  individual  condition  reported 
that  they  would  take  other  courses  structured  like  this  one,  compared  to  36%  of  students  in  the  cooperative  condition  and 
22%  in  the  collaborative  condition. 

Furthermore,  responses  to  the  interviews  revealed  somewhat  mixed  feelings  for  the  small  group  structures 
implemented  in  this  study.  Results  showed  that  57%  of  individual  subjects,  52%  of  cooperative  subjects,  and  43%  of 
collaborative  subjects  indicated  a preference  for  working  alone  rather  than  with  a partner.  While  more  than  60%  of 
students  in  all  three  conditions  reported  that  they  would  enjoy  working  with  a partner,  less  than  half  of  subjects  in  all 
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conditions  said  they  would  learn  more  working  with  a partner.  This  is  consistent  with  other  researchers  who  found  that 
some  students  like  working  in  small  groups  even  when  their  achievement  does  not  increase  (Palinscar  & Brown,  1979). 

The  results  of  the  current  study  may  be  explained  by  the  elaborations  provided  by  students  during  the  interviews. 
Many  of  the  subjects  from  the  cooperative  and  collaborative  treatments  qualified  their  responses  about  small  group 
structures  with  the  conditions  that  1)  their  partner  should  be  a student  who  had  a similar  commitment  level  to  school,  2) 
the  subject  should  require  thought  and  analysis  rather  than  memorization,  and  3)  the  class  format  should  include  in-class 
activities  and  instructor  assistance  on  team  formation  and  skills.  They  thought  such  a course  must  have  "hard  stuff,  not 
self-explanatory  stuff  or  stuff  you  just  memorize"  and  that  a "lab  class  isn't  good  (for  small  group  structures)  unless  the 
computers  are  networked  and  you  change  the  software." 

Results  from  the  interviews  also  suggest  that  approximately  44%  of  subjects  in  both  small  group  structures  did 
not  consistently  use  the  workbooks  which  were  designed  to  prompt  them  to  c<  operate  or  collaborate.  Observation  of 
students  in  the  small  groups  provided  evidence  that  subjects  used  the  workbook  during  Day  1 of  the  study,  but  tended  to 
ignore  the  workbook  on  subsequent  days.  Students  who  elaborated  their  answer  to  the  interview  questions  provided 
further  evidence  that  the  workbooks  were  not  implemented  as  intended.  Many  students  remarked  that  they  only  used  the 
workbook  to  keep  track  of  where  they  got  to  each  day. 

Even  though  they  did  not  use  the  workbooks  as  directed,  students  in  the  small  group  conditions  did  exhibit 
interaction  behaviors  during  the  lesson.  Using  the  workbooks  during  Day  1 of  the  lesson  may  have  been  enough  to 
prompt  these  interactions.  Prior  experience  in  small  group  learning  in  the  introductory  accounting  course  also  may  have 
contributed  to  the  occurrence  of  these  interaction  behaviors. 

The  results  for  student  interaction  were  consistent  witl^the  design  of  the  cooperative  and  collaborative  learning 
structures.  Students  in  the  cooperative  dyads  exhibited  sign^cantly  more  discussion  about  the  content  and  tasks  than 
students  in  the  collaborative  dyads.  Students  in  the  cooperative  structure  also  provided  answers  to  their  partners' 
questions  significantly  more  often  than  those  in  the  collaborative  structure. 

These  differences  can  be  attributed  to  the  nature  of  the  two  small  group  structures  under  investigation.  In  the 
cooperative  structure,  students  were  told  to  assume  specific  roles  of  preparer  and  checker  arid  to  switch  these  roles 
frequently.  Cooperative  subjects  also  used  one  computer  throughout  the  lesson.  In  keeping  with  Davidson's  (1985) 
suggestions  for  how  to  design  collaborative  learning,  the  collaborative  dyads  were  provided  with  less  structure  than  the 
cooperative  dyads.  Studer.ts  in  the  collaborative  dyads  each  used  their  own  computer  and  were  directed  "not  to  discuss  as 
you  go"  but  rather  to  wait  and  consult  with  their  partner  about  their  questions  at  specific  points  in  the  lesson. 

Observations  of  each  dyad  for  five  minute  intervals  on  the  last  day  of  the  study  confirmed  that  students  in  the 
cooperative  and  collaborative  conditions  exhibited  different  patterns  of  interactions.  Many  students  in  the  cooperative 
learning  structure  talked  aloud  while  working  through  the  simulation,  asked  questions,  and  waited  for  agreement  from 
their  partner.  Most  exchanges  of  information  went  back  and  forth  many  times,  as  opposed  to  one  sided  comments. 

Many  students  in  the  cooperative  dyads  looked  at  each  other  when  they  spoke. 

In  contrast,  students  in  the  collaborative  learning  structure  seemed  to  develop  a non  verbal  signal  of  staring  at 
their  partner's  screen  when  they  needed  to  consult.  Little  eye  contact  or  face  to  face  conversations  occurred.  Most 
collaborative  students  spoke  to  their  own  computer  screens  when  asking  questions  or  giving  answers.  Comments  were 
usually  brief,  one  sided  events. 

The  current  study  provides  some  implications  for  educational  technologists  who  design  and  implement 
computer-based  instruction.  Results  indicate  that  small  group  structures  do  not  always  increase  student  learning  and 
motivation,  especially  for  high  functioning  students.  When  computer  resources  permit  each  student  to  work  at  a 
computer,  the  investment  of  additional  time  to  create  small  group  learning  structures  may  not  be  justified.  In  cases 
where  learning  objectives  specifically  include  the  mastery  of  group  and  social  skills,  the  investment  of  time  necessary  to 
redesign  the  computer  instruction  for  group  use  may  be  warranted.  This  study  suggests  that  adjunct  materials  such  as 
workbooks  are  not  the  best  method  of  prompting  students  to  work  together  when  using  computer  instruction. 
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Since  computer  resources  are  often  limited,  educators  are  forced  to  group  several  students  at  one  computer. 

Future  research  should  be  conducted  to  determine  the  most  effective  ways  to  enhance  learning  when  students  work  in 
small  groups  at  the  computer.  A recent  study  by  Sherman  & Klein  (in  press)  indicates  that  embedded  prompts  can 
influence  student  interactions  and  achievement  when  cooperative  learning  is  combined  with  computer  instruction.  Future 
research  could  explore  embedded  prompts  in  computer  simulations.  Research  should  also  continue  to  examine  different 
types  of  learning  structures  when  students  use  computer  technologies.  Finally,  future  research  should  continue  to  explore 
the  use  of  computer  simulations  in  college  classrooms  to  determine  the  most  effective  structures  for  implementing  this 
technology  into  the  curriculum.  Such  research  can  assist  educational  technologists  in  constructing  the  best  combination 
of  learning  structure  and  tasks  so  that  the  learning  experience  is  enhanced. 
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Table  1 


Number  of  Student  Interactions  for  Cooperative  and  Collaborative  Dyads 


Learning  Structure 


Type  of  Interaction 


Cooperative  Collaborative 


Questioning 

181 

157 

Answering 

194 

136 

Encouraging 

61 

49 

Discussing 

459 

218 

Off  task 

21 

21 

Table  2 

Summary  of  Student  Responses  to  Interview  Questions 


What  features  of  this  computer  lesson  helped  you  learn? 


Ind 

Coop 

Coll 

Simulated  practice  sets 

79% 

48% 

57% 

Working  with  a partner 

0% 

24% 

40% 

Booklets 

18% 

0% 

0% 

Everything 

3% 

28% 

3% 

What  features  of  the  student  booklet  did  you  find 

helpful? 

Ind 

COOD 

Coll 

Didn't  use  booklet 

57% 

45% 

43% 

Organization/instructions 

36% 

55% 

43% 

All  of  it 

7% 

0% 

14% 

Do  you  prefer  to  work  alone  or  with  a partner  to  learn? 

Ind 

Coop 

Coll 

Alone 

57% 

52% 

43% 

Partner 

43% 

38% 

40% 

Either  is  OK 

0% 

10% 

17% 

Do  you  think  you  would  have  learned  more  if  you  had  worked  alone  or  with  a partner? 

M Cgqr 


Coll 


Alone 

43% 

21% 

Partner 

46% 

38% 

Same  with  either 

11% 

38% 

Depends  on  subject 

0% 

3% 

27% 

40% 

33% 

0% 


Do  you  think  you  would  have  enjoyed  this  module  more  if  you  had  worked  alone  or  with  a partner? 


Alone 

Partner 

Same  with  either 


Ind Coop Coll 


29% 

31% 

37% 

71% 

62% 

60% 

0% 

7% 

3% 
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Overview 


A careful  review  of  the  teacher  planning  literature  illustrates  clearly  that  teachers  rely  on  mental  planning  to 
guide  what  occurs  in  their  classrooms  . This  mental  planning,  or  reflective  mental  dialog,  not  only  precedes  written 
planning,  but  occurs  throughout  the  design,  implementation,  and  evaluation  phases  of  instruction.  It  is  a key  element  in 
planning  prior  to  teaching,  in  reflection,  monitoring,  and  adjustment  during  teaching,  and  in  reflective  evaluation  and 
revision  following  teaching. 

This  presentation  builds  upon  the  findings  (often  mixed)  of  prior  research  , and,  based  upon  the  results  of 
ongoing  studies  of  the  practices  of  elementary  school  teachers,  focuses  on  the  teacher  use  of  ID  skills  in  the  planning 
and  delivery  of  instruction.  In  particular  it  takes  a close  look  at  the  relative  and  "real"  use  of  ID  practices  in  both  mental 
and  written  planning. 

Background 

Two  decades  ago,  Beilby’s  (1974)  efforts  to  narrow  the  gap  between  teacher  educators  and  instructional 
technologists  focused  on  our  need  as  a field  to  “relinquish  a considerable  portion  of  [our]  ID  role  to  teachers”  (p.  12).  His 
charge  was  based  on  our  common  mission  “to  facilitate  and  improve  the  quality  of  human  learning”  (Ely  etal , 1972). 
Although  this  clarion  call  was  repeated  by  Stolovitch  (1980),  very  little  change  occurred  until  the  decade  of  criticism 
spawned  by  A Nation  at  Risk  (1983). 

In  the  more  inviting  context  of  school  reform,  instructional  designers  began  to  display  increasing  interest  in 
sharing  their  skills  with  classroom  teachers.  The  proposed  interventions  covered  the  whole  gamut  of  the  educational 
process  — from  teacher  preparation  (Burkman,  1987;  Earle,  1985,  1992;  Klein,  1991;  Martin  & Clemente,  1990;  Reiser 
& Mory  1991 ; Reiser  & Radford,  1990;  Snelbecker,  1987)  to  teacher  inservice  (Shrock  & Byrd,  1987;  Schiffman,  1987) 
to  systems  redesign  (Banathy,  1991;  Branson,  1987;  Reigeluth,  1987;  Reigeluth  & Garfmkle,  1992;  Reiser  & Salisbury, 
1991;  Salisbury,  1993).  However,  the  approach  to  reconciliation  was  often  one-sided,  more  along  the  lines  of  what  we 
can  offer  you,”  rather  than  a search  for  common  ground  — an  approach  which  re-emphasized  the  fact  that  these  two 
professional  groups  have  remained  separate  and  aloof  in  both  research  literature  and  instructional  theories. 

Although  it  might  be  reasonable  to  assume  that  researchers  and  theorists  from  two  closely  related  fields  such  as 
teacher  education  and  instructional  design  would  work  collaboratively  to  exchange  ideas  and  concerns  about  the 
improvement  of  instruction,  the  gap  between  both  fields  has  instead  widened  over  the  years.  Until  recently,  instructional 
design  and  teacher  education  were  viewed  as  separate  fields.  Despite  an  amazingly  common  interest  (i.e.,  the  teaching  and 
learning  process),  teacher  educators  and  instructional  designers  read  and  write  separate  literature  and  study  different 
theoretical  procedures  . 


Hence  ID  interventions  for  the  classroom  are  perceived  by  teachers  and  teacher  educators  as  low  in  credibility 
largely  because  there  exists  little  common  language  or  understanding  for  communication.  If  we  are  to  successfully 
continue  our  venture  into  school  reform,  then  we  need  to  understand  what’s  happening  in  schools  and  in  teacher 
education. 

All  of  us  who  wish  to  contribute  to  schools  would  be  well  advised  to  read  what  school  people  read  and  go  where 
school  people  go.This  means  stretching  beyond  our  typical  spheres  of  communication  to  try  to  understand  schooling 
from  the  perspective  of  those  who  “live”  there  (Shrock,  1990,  p.  29). 

While  many  teacher  educators  view  instructional  design  skills  as  important,  few  teacher  education  programs  offer 
courses  that  would  provide  opportunities  for  students  to  develop  instructional  design  skills.  Since  it  is  unlikely  that  the 
public  schools  will  employ  many  instructional  designers,  and  it  is  likely  that  teachers  in  public  schools  will  pay  more 
attention  to  design  principles,  teacher  education  programs  are  challenged  to  develop  strategies  to  bridge  the  gap  between 
the  theory  of  instructional  design  and  fhe  practice  of  teaching. 


Recent  efforts  by  several  instructional  designers,  in  providing  focused  observations  on  what  teachers  do  in  their 
classrooms  and  how  we  can  best  mesh  our  expertise  with  theirs,  have  generated  the  following  insights: 

• systems  approach  principles  can  be  taught  to  preservice  teachers 

• differences  exist  between  Id  models  and  teacher  models  of  thinking  and  learning 

• teachers  implicitly  apply  ID  practices  when  planning  to  teach 

• teachers  think  and  talk  about  instructional  planning  in  different  ways 

Reiser  (1994)  reflected  upon  three  years  of  research  into  the  use  of  ID  skills  in  teacher  planning,  sharing  his 
overall  impressions  as  well  as  suggestions  for  future  research.  He  indicated  that  systems  approach  principles  can  be 
taught  to  preservice  teachers  quite  successfully  but  that  additional  instruction  and  skill  applications  were  necessary  to 
ensure  effective  and  continued  use  of  ID  skills  by  teachers.  Moallem  (1994)  explored  an  expert  teacher’s  model  of 
thinking  and  teaching  within  the  context  of  the  social  context  of  the  classroom.  This  ethnographic  study,  which 
compared  teacher  thinking  to  instructional  design,  suggested  fundamental  differences  between  these  two  models.  Kennedy 
(1994),  in  order  to  determine  whether  teachers  use  ID  or  personal  heuristics,  reviewed  four  Canadian  studies  of  ID 
knowledge,  competency,  and  use.  She  concluded  that  teachers  have  an  abysmal  understanding  of  learning  theory  as  an 
underlying  framework  for  instruction.  She  felt  that  their  inability  to  see  instruction  from  a systems  perspective  was 
ample  evidence  of  the  need  for  ID  skills  in  preservice  training. 

Branch  (1994)  introduced  his  study  of  secondary  teachers  with  a discussion  of  the  constituent  elements  of  an 
instructional  episode  and  the  fundamental  components  of  ID.  In  particular,  he  stressed  the  need  to  translate  ID  jargon  into 
teacher  language,  and  indicated  that  teachers  implicitly  apply  ID  practices  when  planning  to  teach.  Driscoll,  Klein,  and 
Sherman  (1994)  examined  how  teachers  and  instructional  designers  think  and  talk  about  instructional  planning.  They 
explored  the  differences  which  exist  in  how  both  groups  conceive  of  their  practices  in  order  to  determine  how  such 
differences  contribute  to  the  lack  of  perceived  impact  of  ID  in  teaching  contexts.  Garbosky  (1994)  took  a ‘‘then  and  now” 
approach  by  comparing  (after  six  years)  the  activities,  experience,  and  feelings  of  educators  formally  trained  in 
instructional  design.  Her  article  illustrated  quite  well  the  reality  and  the  breadth  of  the  gap  between  the  two  fields. 

There  is  no  question  that  teachers  design  instruction,  even  if  they  do  not  follow  classical  ID  principles,  and  even 
if  they  don't  consider  what  they  do  to  be  instructional  design  ( Clark  & Angert,  1980).  Zahorik  (1975)  classified  teacher 
decisions  in  the  design  process  into  eight  categories:  objectives,  content,  activities,  materials,  diagnosis,  evaluation, 
instruction,  and  organization.  Note  the  similarities  between  these  categories  of  design  decisions  and  the  common 
elements  of  ID  models  compiled  by  Andrews  and  Good?  >n  (1980).  In  addition  to  these  categories,  the  teacher  planning 
literature  has  attended  to  the  time  frames  of  planning  and  the  products  or  processes  of  planning  (McCutcheon,  1980; 

Clark  & Yinger,  1979;  Morine-Dersbimer,  1978-9;  Peterson,  Marx,  & Clark,  1978;  Zahorik,  1975).  More  recent 
research  has  emphasized  the  practical  application  of  ID  skills  in  the  planning  processes  of  teachers  (Earle  1992;  Reiser  & 
Mory,  1991;  Klein,  1991;  Martin,  1990;  Martin  & Clemente,  1990).  Research  on  teacher  planning  emphasizes  its 
importance  as  process  (Arnold,  1988;  Yinger  1977)  and  indicates  that  teachers  make  decisions  "about  lesson  plans, 
interactive  teaching,  modifications  required  during  teaching,  and  other  ways...  to  routinely  plan  and  evaluate  and  modify 
instruction"  (Snelbecker,  1987,  p.  35). 

Bridging  the  Gap:  Two  Studies 

Do  teachers  typically  employ  instructional  design  practices  when  they  are  planning  their  instruction?  If  so, 
what  practices  do  they  employ?  If  not,  why  don't  they  employ  them?  Are  the  planning  practices  of  those  pre-service  and 
in-service  teachers  who  have  been  taught  basic  instructional  design  principles  different  from  the  practices  ot  those  who 
have  not  been  taught  these  principles? 

The  first  study  involved  twenty-two  elementary  teachers  from  schools  across  North  Carolina  (NC)  The  second 
study  included  seventeen  elementary  teacher  from  Provo  School  District  in  Utah  (UT).  Both  groups  responded  to  a 
detailed  survey  which  covered  demographics,  general  information,  and  practices  in  yearly,  unit,  and  daily  planning. 
Similar  questions  addressed  each  level  of  planning.  Follow-up  interviews  which  delved  further  into  planning  issues  were 
held  with  teachers  selected  from  each  group. 
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Patterns  of  Practice 


The  results  of  both  studies  with  elementary  school  teachers  indicate  the  following  trends  or  patterns  in  their 
planning  and  delivery  practices: 

• Teachers  favored  mental  planningwhile  recognizing  the  importance  of  written  planning,  particularly  at  the  unit  level. 
Production  of  written  plans  was  closely  related  to  administrative  requirements. 

• Plans  tended  to  be  more  specific  at  the  unit  and  daily  levels  in  all  areas  (content,  materials,  activities,  objectives,  and 
tests)  and  more  general  at  the  yearly  level  (see  Table  1 1). 

• Most  teachers  who  have  had  formal  training  in  instructional  design  felt  that  a knowledge  of  ID  had  improved  their 
planning  processes  (NC  81%  ; UT  90%). 

• Most  teachers  consciously  used  ID  processes  in  their  planning  (NC  60%  ; UT  73%). 

• The  crucial  elements  of  the  ID  process  were  goals,  learner  analysis,  objectives,  activities  and  strategies,  tests,  and 
revision  (see  Table  4).  These  aspects  were  also  considered  formally  (see  Table  1)  and  were  more  likely  to  be  included 
in  written  plans  (see  Table  2).  Note  the  mixed  results  with  learner  analysis.  Task  analysis,  types  of  learning, 
instructional  plans,  and  try-out  were  considered  helpful  but  tended  to  be  addressed  informally  and  mentally. 

• ID  processes  received  more  attention  at  the  unit  and  daily  levels  of  planning  rather  than  at  the  yearly  level  (see  Table 

3). 

• Most  teachers  gave  equal  importance  to  written  and  mental  planning  (see  Table  5). 

• During  teaching,  there  was  less  deviation  from  unit  and  daily  plans  (both  of  which  were  considered  more  important) 
than  from  yearly  plans  (see  Tables  6 and  7). 

• Initial  planning  decisions  centered  around  content  and  objectives  at  all  levels  (see  Table  10)  while  most  planning 
time  focused  on  content,  materials,  and  activities  (see  Table  9). 

• Trying  out  the  instruction  prior  to  using  it  in  the  classroom  was  impractical  for  elementary  teachers.  They  relied  on 
mental  imagery  and  planning  to  test  the  instruction  prior  to  delivery.  Afterwards,  reflection  allowed  for  revision. 

Moving  Ahead* 

Instructional  designers  offer  teachers  a vast  array  of  expertise  in  the  improvement  of  instruction  and  learning. 
However,  since  teachers  and  technologists  often  view  both  teaching  and  technology  differently,  emphasis  must  be  placed 
on  closing  the  gap,  on  developing  productive  ways  of  working  togerher.  "In  order  to  make  any  real  headway  as  a field, 
we  will  have  to  come  to  some  reconciliation  of  our  differences  if  we  want  our  diffusion  efforts  in  teacher  education  to  pay 
off  (Martin,  1990,  p.  56). 

However  the  pay-off  will  depend  on  our  performance.  In  an  analysis  of  the  techniques  of  Coach  "Bear"  Bryant  in 
building  a winning  team  at  the  University  of  Alabama,  Gilbert  and  Bilbert  (1988)  report: 

Two  -thirds  of  all  fourth  grade  kids  who  sign  up  voluntarily  to  learn  to  play  woodwind  instruments  quit 
within  60  days.  The  reason?  No  indication  of  any  likelihood  of  success...  when  it  comes  to 
motivation,  phony  displays  of  warmth  are  no  substitute  for  evidence  of  successful  performance  (p.  34). 


1 These  ideas  were  first  expressed  in  Earle,  R.S.  (1994'.  Instructional  design  and  the  classroom  teacher:  Looking  back  and 
moving  ahead.  Educational  Technology , 34  (3),  6-10 
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May  I suggest,  then,  that  as  we  move  ahead  to  where  we,  as  instructional  designers,  should  be  in  our 
relationship  with  classroom  teachers  and  the  public  schools  we  base  our  performance  on  the  following  guidelines  derived 
from  looking  back  at  where  we've  been. 

• Develop  a common  technical  language  of  instruction  based  on  an  integration  of  the  literature  from  both  fields. 

• Validate  the  scientific  bases  of  teaching  as  essential  precursors  of  the  art  of  teaching. 

• Adopt  a layers-of-necessity  philosophy  in  modifying  classical  ID  to  meet  the  needs  and  practices  of  teachers. 

• Recognize  the  need  for  piece-meal  reform  and  fundamental  systemic  restructuring  as  concurrent,  interactive  ventures. 

An  examination  of  the  mental  planning  processes  of  teachers  necessarily  brings  us  face  to  face  with  the 
multifaceted,  dynamic,  complex  nature  of  the  teaching.  Though  the  process  is  often  perplexing,  we  have  an  opportunity 
to  savor  the  adventure  of  taking  a look  into  the  world  of  the  classroom  teacher.  As  we  strive  to  blend  the  research  and 
theories  of  teacher  education  and  instructional  design,  perhaps  a deeper  recognition  of  the  richness  of  teacher  planning 
practices  will  provide  avenues  for  the  modified  implementation  of  ID  strategies  and  techniques  to  enhance  the  quality  of 
human  learning. 


Table  Is  Formal  and  Informal  Use  of  ID  Processes  (%) 

Formal  • Informal 

NC  UT  NC  UT 


Develop  or  review  course  and  unit  goals 

43 

63 

48 

50 

Develop  a task  analysis  or  learning 
hierarchy  to  identify  prerequisite  skills 
and  sequence  of  instruction 

24 

25 

67 

68 

Classify  types  of  learning  indicated  in 
the  content 

14 

12 

76 

88 

Analyze  the  abilities  and  need  of 
learners 

71 

25 

24 

88 

Develop  performance  or  behavioral 
objectives 

62 

50 

38 

56 

Develop  tests  that  match  the  learnings 
described  in  the  objectives 

62 

69 

33 

38 

Select  or  produce  learning  activities  and 
strategies  that  match  the  type  of 
learning  and  objective 

76 

69 

29 

31 

Follow  a systematic  instructional  plan 
(e.g.,  Gagne’s  events  of  instruction  or 
Madeline  Hunter's  steps,  etc.) 

62 

31 

43 

63 

Try  out  the  instruction  prior  to  using  it 
in  the  classroom 

14 

0 

71 

81 

Revise  the  instruction  based  on  the 
results  observed  during  teaching 

52 

56 

51 

69 
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Table  2:  Written  or  Mental  Plans  (%) 


Written  Mental 

NC  UT  NC  UT 


Goals 

62 

88 

62 

63 

Task  Analysis 

29 

63 

76 

69 

Tvoes  of  Learning 

14 

6 

90 

88 

Learner  Analysis 

43 

31 

86 

88 

Objectives 

86 

75 

33 

38 

Tests 

86 

81 

52 

50 

Activities  & Strategies 

86 

63 

52 

56 

Instructional  Plan 

52 

69 

62 

81 

Try  Out 

19 

0 

86 

81 

Revise 

76 

50 

71 

81 

Table  3:  Use  of  ID  Processes  in  Yearly,  Unit,  and  Daily  Planning  (%) 


Year  Unit  Day 


NC 

UT 

NC 

UT 

NC 

UT 

Goals 

50 

63 

70 

81 

30 

25 

Task  Analysis 

30 

38 

45 

69 

25 

38 

Types  of  Learning 

10 

25 

25 

63 

35 

31 

•Learner  Analysis 

30 

44 

50 

50 

65 

75 

Objectives 

40 

38 

45 

81 

45 

63 

Tests 

10 

13 

55 

88 

35 

50 

Activities  & 
Strategies 

■ 20 

13 

55 

88 

70 

69 

Instructional  Plan 

10 

13 

35 

56 

70 

69 

Try  Out 

10 

6 

20 

19 

25 

44 

Revise 

30 

6 

55 

69 

75 

81 

Table  4:  The  Value  of  ID  Processes  (%) 

Year  Unit  Day 


NC 

UT 

NC 

UT 

NC 

UT 

Goals 

67 

88 

33 

12 

0 

0 

Task  Analysis 

25 

25 

67 

75 

8 

0 

Types  of  Learning 

17 

12 

50 

88 

33 

0 

Learner  Anal}  sis 

92 

88 

8 

12 

0 

0 

Objectives 

83 

5 

0 

25 

17 

0 

Tests 

75 

69 

17 

31 

8 

0 

Activities  & Strategies 

83 

69 

17 

31 

0 

0 

Instructional  Pkan 

17 

19 

75 

69 

8 

12 

Try  Out 

0 

0 

75 

75 

25 

25 

Revise 

67 

63 

33 

37 

0 

0 
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Table  5:  Importance  of  Written  and  Mental  Plans  (%) 


Written  Mental  Equal 


NC 

UT 

NC 

UT 

NC 

UT 

Overall 

25 

6 

37.5 

19 

37.5 

75 

Year 

31 

53 

31 

7 

38 

40 

Unit 

29 

19 

7 

12 

64 

69  1 

Daily 

33 

0 

27 

31 

40 

69 

Table  6:  Following  Plans  (%) 


Year  Unit  Daily 


NC 

UT 

NC 

UT 

NC 

UT 

Very  Closely 

8 

7 

8 

6 

13 

20 

Closely 

(<  25%  deviation) 

46 

33 

77 

69 

67 

67 

Somewhat  Closely 
(25-49%  deviation) 

31 

60 

15 

25 

20 

7 

Somewhat  Loosely 
(50-75%  deviation) 

15 

0 

0 

0 

0 

6 

Very  Loosely 
(>  75%  deviation) 

0 

0 

0 

0 

0 

0 

Table  7:  Importance  of  Planning  (%) 


Year  Unit  Daily 


NC 

UT 

NC 

UT 

NC 

UT 

Crucial 

(100%  of  the  time) 

23 

20 

31 

38 

60 

33 

Useful 

(75%  of  the  time) 

46 

60 

69 

62 

40 

67 

Generally  Useful 
(50%  of  the  time) 

31 

13 

0 

0 

0 

0 

Minimally  Useful 
(25%  of  the  time) 

0 

7 

0 

0 

0 

0 

Not  Very  Useful 
(<10%  of  the  time) 

0 

0 

0 

0 

0 

0 

Table  8:  Amount  of  Written  Planning  (%) 


Overall  Yearly  Unit  Daily 


NC 

UT 

NC 

UT 

NC 

UT 

NC 

UT 

More  than  75% 

29 

19 

33 

44 

33 

19 

27 

19 

50%-74% 

18 

31 

20 

12 

33 

44 

20 

12 

25%-49% 

35 

25 

27 

19 

27 

25 

33 

50 

Less  than  25% 

18 

25 

20 

25 

7 

12 

20 

19 
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Table  9:  Percentages  of  Planning  Time  (Mean  %) 


Yearly  Unit  Daily 


NC 

UT 

NC 

UT 

NC 

UT 

Content 

26 

32 

22 

21 

18 

23 

Materials 

17 

20 

19 

22 

24 

28 

Activities 

23 

20 

27 

27 

28 

25 

Objectives 

24 

17 

20 

15 

20 

13 

Tests 

7 

9 

10 

12 

9 

10 

Table  TO:  First  Decisions  (%) 


Yearly  Unit  Daily 


NC 

UT 

NC 

UT 

NC 

UT 

Content 

43 

64 

29 

71  1 

14 

29 

Materials 

5 

0 

5 

0 

5 

7 

Activities 

5 1 

0 

0 

0 

14 

14 

Obiectives 

48 

36 

62 

29 

67 

50 

Tests 

0 

0 

0 

0 

5 

0 

Table  11:  Specificity  of  Plans  (%):  NC 


Very  Specific 

Somewhat  Specific 

Somewhat  General 

Very  General 

Y 

U 

D 

Y 

U 

D 

Y 

u 

D 

Y 

u 

D 

Content 

30 

44 

37 

15 

44 

47 

35 

11 

n 

20 

0 

0 

Materials 

20 

44 

32 

25 

39 

47 

30 

11 

16 

20 

6 

5 

Activities 

15 

33 

37 

15 

56 

42 

45 

17 

21 

20 

0 

0 

Objectives 

40 

61 

47 

40 

33 

47 

10 

6 

0 

10 

0 

11 

Tests 

10 

22 

32 

20 

39 

32 

10 

17 

o ! 

35 

17 

20 
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Table  12:  Specificity  of  Plans  (%):  UT 


Very  Specific 

Somewhat  Specific 

Somewhat  General 

Very  General  ! 

Y 

U 

D 

Y 

u 

D 

Y 

U 

D 

Y ] 

u 

D 

Content 

20 

50 

38 

27 

38 

56 

40 

12 

6 

13 

0 

o ! 

Materials 

20 

38 

56 

40 

62 

38 

33 

0 

6 

7 
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0 1 

Activities 

13 

3i 

38 

47 

63 

56 

20 

6 

6 

20 
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0 

Objectives 

53 

69 

60 

20 

25 

27 

7 

6 

13 

20 

0 

0 

Tests 

27 

38 

53 

20 

56 

33 

33 

6 

13 

20 

0 

0 ! 
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Abstract 


College  courses  have  now  gone  beyond  the  walls  of  the  traditional  college  classroom  into  cyberspace.  These  new 
Internet  course  offerings  bring  about  major  instructional  design  issues  that  must  be  acknowledged  and  addressed  during  the 
course  development  phase.  Two  of  the  major  issues  are  Internet  access  requirements  and  clarity  of  instruction.  However, 
there  are  also  other  educational  issues  that  must  be  considered  such  as  incidental  learning,  quality  control,  and  cultural 
climate. 


Internet-Distributed  College  Courses: 

Instructional  Design  Issues 

Moving  the  campus  to  the  cyberspace  horizons  has  forced  course  designers  to  rethink  their  instructional  strategies 
and  delivery  methods.  We  no  longer  think  of  single  classrooms,  but  interlinked  students  both  on  campus  and  globally. 
Choices  need  to  be  made  as  to  the  target  audience  and  whether  or  not  it  will  be  a university-wide  virtual  offering  or  a 
cyberspace,  international  virtual  offering.  As  we  enroll  diverse  students  from  a variety  of  nations,  our  instructional 
strategies  have  to  break  the  mold  associated  with  traditional,  on-site,  classroom-contained  teaching  methods  (Shuell, 
1993).  As  a result,  ease  of  access  to  the  Internet  and  clarity  of  instruction  have  become  two  paramount  issues  that  must 
be  dealt  with  at  the  onset  of  the  course  development  process.  Other  educational  issues  also  surface  during  course 
construction  and  delivery. 


Internet  Access 

Internet  access  was  one  of  the  initial,  major  concerns  that  faced  the  authors.  Although  through  the  university’s 
system,  access  capabilities  typically  included  e-mail,  gophers,  telnet,  and  the  World  Wide  Web,  not  everyone  who 
enrolled  in  the  courses  would  necessarily  have  access  to  the  university’s  system  or  to  all  the  levels  of  the  Internet.  Thus, 
an  examination  was  done  as  to  the  capabilities  on  different  systems. 

For  the  most  part,  it  was  determined  that  individuals  who  would  enroll  in  the  Internet-distributed  college  courses 
would  have  accounts  on  a university  system,  a commercial  Internet  provider,  a local  school  district  system,  or  a local 
freenet  system.  Levels  of  Internet  access  varied  with  each  of  those  systems.  With  university  systems  and  commercial 
Internet  providers,  access  to  the  Internet  seemed  to  be  the  most  complete.  Next  in  line  was  a freenet  system;  however, 
some  freenet  systems  did  not  have  telnet  or  Web  capabilities.  Internet  access  via  local  school  district  systems  seemed  to 
be  the  most  limiting,  since  many  systems  allowed  for  only  e-mail  access. 

After  the  Internet  access  analysis  was  completed,  the  next  thing  that  the  authors  did  was  decide  on  the  Internet- 
related  activities  that  they  wanted  the  students  to  perform.  In  cases  where  the  course  consisted  of  information  distribution 
and  communication  between  peers  and  instructor,  basic  e-mail  access  was  all  *hat  was  needed.  However,  in  the  courses 
that  required  the  enrollees  to  search  the  Internet  for  course-related  information  and  material,  e-mail  only  Internet  access 
became  a major  obstacle.  To  prevent  problems  caused  by  people  enrolling  in  courses  and  then  later  finding  out  that  their 
Internet  access  was  not  sufficient  to  complete  the  Internet  course,  the  decision  was  made  to  include  requisite  Internet 
access  capabilities  as  part  of  the  course  description. 

Clarity  of  Instruction 

Instructor  Presence 

Clarity  of  instruction  was  the  second  key  factor  that  caused  concern  for  the  authors.  Professor  and  student 
“presence”,  in  the  traditional  sense,  was  virtually  eliminated  with  the  Internet  distributed  courses.  Thus,  the  protessor  had 
to  clearly  define  the  course  content,  while  identifying  all  possible  interpretations  of  what  was  actually  included  in  the 
context  of  the  information  sent  to  the  students.  In  an  attempt  to  replace  the  professor-initiated,  in-class  reinforcement, 
learning  objectives  had  to  be  correlated  and  specifically  communicated  to  the  students  for  each  course  topic  covered  and 
assignment  given  (Shuell,  1996).  This  was  done  to  alleviate  any  misunderstanding  and  to  show  an  interconnection 
between  the  course  objectives,  the  topics  covered,  the  assignments,  and  the  use  of  the  Internet.  As  a result,  course 
preparation  time  and  effort  increased  because  of  the  expanded  clarification  demands. 


Levels  of  Interactivity 

Not  only  did  the  course  information  have  to  be  clear  and  concise,  the  level  of  interactivity  and  quantity  of 
communication  were  associate  issues.  It  was  important  to  consider  the  amount  of  interpersonal  exchange  necessary  to 
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meet  the  course  objectives  (Shuell,  1993).  Each  course  had  its  own  level  of  interactivity.  There  were  four  levels  of 
interpersonal  exchange  that  entered  into  the  design  process:  instructor-student,  student-student,  instructor-guest  expert, 
student-guest  expert.  A decision  had  to  be  made  as  to  the  synchronous  versus  asynchronous  communication  that  needed 
to  occur  at  each  interpersonal  exchange  level.  To  enhance  professor-student  intercourse,  electronic  office  hours  were 
established  to  create  an  atmosphere  somewhat  resembling  the  traditional  office  time  required  of  professors  teaching  on- 
campus. 

Course  Enrollment 

Course  enrollment  was  yet  another  factor  that  influenced  the  communication  exchange  expected  to  take  place 
during  a virtual  course.  For  courses  that  have  a large  enrollment  (e.g.,  over  50  course  enrollees),  the  quantity  of 
synchronous  communication  has  to  be  either  severely  limited  or  eliminated.  Even  for  courses  where  the  number  of 
enrollees  is  fewer  than  50,  guidelines  need  to  be  formulated  before  the  start  of  the  course  in  an  effort  to  keep  the 
communication  flowing,  but  manageable  (Shuell,  1996). 

Other  Educational  Issues 

There  were,  and  are,  many  educational  issues  associated  with  Internet-distributed  coursework.  Decreased  incidental 
learning  from  cyberspace  isolation  must  be  considered  not  only  in  the  development  process  but  by  the  potential  student. 
Quality  control  issues  arise  when  there  are  no  cyberspace  proctors  to  validate  a course  enrollee’s  identification;  do  you 
know  that  the  student  who  is  receiving  the  grade  did  the  work?  There  are  cultural  climate  issues  both  in  the  development 
and  implementation  of  the  course  related  to  instructor  and  student  input,  terminology,  interpretation,  and  overall 
professional  conduct.  As  university  faculty  and  administrators  become  more  experienced  in  offering  virtual  courses,  the 
easier  it  will  be  to  research  educational  issues  and  formulate  solutions  to  the  virtual  problems  that  arise.  Unfortunately, 
we  are  still  struggling  with  these  major  educational  issues,  but  that  does  not  preclude  their  consideration  and 
identification  since  those  issues  can  impact  the  overall  integrity  of  the  Internet-distributed  course. 

Conclusion 

As  with  all  teaching,  Internet-distributed  courses  need  to  be  constantly  evaluated  and  revised  to  continually 
improve  the  product  and  maintain  the  quality  reputation  of  the  professor  and  the  offering  institution.  Conquering  this 
new  frontier  is  difficult  and  will  remain  challenging,  but  in  today’s  society,  universities  who  do  not  venture  into  this  vast 
electronic  world  face  the  possibility  of  losing  students  to  virtual  competitors. 
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Most  books  are  stimulated  by  events  or  needs  experienced  by  individuals.  The  preparation  of  Classic  Works  on 
Instructional  Technology  was  no  different.  The  editors  participated  in  the  design  of  an  educational  technology  curriculum 
at  the  University  of  Twente  in  The  Netherlands  in  the  early  1980s.  In  an  attempt  to  describe  the  content  of  a new 
curriculum,  a review  of  the  literature  was  necessary.  Questions  about  the  "core"  literature,  "seminal"  works  and  the 
"roots"  of  the  field  led  to  a search  that  continues  today. 

About  the  same  time,  one  of  the  editors  was  teaching  a course  on  Perspectives  of  Educational  Technology,  a 
graduate  seminar  that  attempted  to  communicate  the  "geography"  of  the  field-the  conceptual  contributions  of  the  people, 
the  events,  the  legislation,  the  ideas,  and  the  movements  that  have  helped  to  shape  the  profession.  Logical  resources  for 
such  an  endeavor  are  the  publications  of  the  field-the  written  history  of  the  profession.  The  next  step  is  to  ask:  "What 
publications  are  the  ’classics’  of  the  field-those  books  and  articles  that  have  withstood  the  test  of  time  and  are  still  read 
and  quoted?"  Mark  Twain  described  a classic  as  "something  everyone  wants  to  have  read  and  nobody  wants  to  read.” 

What  would  happen  if  you  were  to  create  a list  of  classics? 

Even  when  a reasonably  satisfactory  list  is  created,  how  does  one  locate  the  articles  themselves,  many  of  which 
have  disappeared  over  the  years?  Sometimes  a colleague  has  squirreled  away  a copy  in  a personal  archive  or  a library  still 
has  the  paper  copy  or  a micofiche.  Wouldn't  it  be  helpful  if  all  these  classic  works  could  be  brought  together  in  one 
volume?  At  least  it  would  save  searching  for  articles  that  are  now  obscure.  If  might  also  help  to  document  the  history 
of  the  field  through  its  literature.  This  book  is  an  attempt  to  make  such  important  articles  readily  available.  It  also 
attempts  to  provide  a partial  conceptual  history  of  the  primary  works  that  often  are  side-stepped  for  more  recent 
interpretations  of  the  older  gems.  When  historians  do  history,  they  go  back  to  the  original  sources,  not  secondary 
interpretations  and  digests.  The  editors  have  attempted  to  collect  the  original  articles  that  are  difficult  to  find.  They  are 
the  original  works~the  primary  writings  upon  which  many  of  today’s  ideas  are  based. 

The  Selection  Process 

What  initially  seems  like  a fairly  routine  task  becomes  a nightmare  because  each  educational  technology 
professional  has  his/her  own  ideas  about  which  articles  are  "classics".  The  process  began  with  one  editor  preparing  a list 
based  on  his  own  experience  and  the  frequency  of  citations  in  major  works  such  as  the  histories  of  Reiser  (1988)  and 
Saettler  (1990)  and  encyclopedia  articles  of  Eraut  (1985)  and  Clark  and  Solomon  (1986).  When  the  second  editor  joined 
the  team,  there  was  agreement  on  more  than  half  of  the  original  titles  and  suggestions  for  alternative  and  additional  titles. 
The  list  was  still  too  long  and  there  was  some  disagreement  about  some  of  the  titles.  Clearly,  a broader  perspective  was 
needed.  A list  of  35  tentative  titles  was  sent  to  40  individuals  who  were  members  of  the  Professors  of  Instructional 
Development  and  Technology  (PIDT)  group  in  the  United  States  and  several  leaders  in  European  countries  who  teach 
courses  in  which  the  publication  might  be  used. 

Respondants  were  asked  to  nominate  other  titles  that  should  be  considered  for  inclusion  the  the  collection.  More 
than  30  individuals  reacted  to  the  original  list  of  titles.  There  was  a high  level  of  agreement  on  about  half  the  titles  and 
suggestions  for  36  more  works,  a half  dozen  of  which  were  mentioned  more  than  once.  Several  new  perceptions  and 
ideas  that  would  influence  selection  emerged  from  the  comments  of  the  respondants: 

* There  are  books  that  ought  to  be  considered  classics.  The  authors  felt  that  a representative  article  or  a chapter  from  a 
book  could  be  included  in  the  collection.  For  example,  Gagne's  Conditions  of  Learning , (1965)  uses  a surrogate  article, 
"Learning  Heirarchies  (Gagne,  1968)  " that  communicates  the  conceptual  essense  of  his  books. 

* North  American  educational  technologists  are  much  less  familiar  with  articles  published  in  Europe;  for  example, 
Davies  (1978).  The  authors  felt  that  their  combined  experience  in  North  America  and  Europe  was  sufficiently  valid  to 
make  judgements  about  selecting  European  articles  even  though  they  were  relatively  unknown  in  North  America. 

* Despite  descriptions  and  instructions  about  the  classics  and  the  criteria  for  expressing  opinions,  many  respondants 
wanted  articles  that  were  more  "up-to-date".  Many  felt  that  there  were  more  recent  publications  that  better  represented 
contemporary  thinking.  For  example,  several  respondants  recommended  using  the  new  Association  for  Education 
Communications  and  Technology  definition  (Seels  and  Richey,  1994)  rather  than  the  1977  AECT  definition.  Of  course 
contemporary  thinking  is  better  represented  by  the  1994  publication  than  the  1977  work  but  it  is  difficult  to  call  a recent 
publication  a classic!  The  authors  had  to  veto  the  suggestions  for  more  contemporary  listings  because  they  would  violate 
the  basic  premise  that  the  articles  are  "...first  to  introduce  the  concept;"  and  "...often  quoted  as  a primary  reference..."  as 
indicated  in  the  list  of  criteria. 
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* The  number  of  new  titles  introduced  by  the  respondants  caused  some  reflection  by  the  authors.  Clearly,  all  of  them 
could  not  be  added  but  some  initial  titles  could  be  dropped  in  favor  of  suggestions  that  seemed  to  make  more  sense. 

Several  changes  were  made  as  a result  of  this  input. 

A further  check  on  the  utility  of  the  proposed  articles  was  their  use  in  a graduate  seminar  held  just  before  the 
final  selection  was  made.  Students  read  and  reacted  to  a wide  spectrum  of  articles,  many  of  which  were  on  the  original 
list.  They  expressed  their  opinions  regarding  the  value  to  them  as  they  were  about  to  enter  the  profession.  Their 
opinions  added  useful  input  and  a "reality  check"  since  they  represented  the  ultimate  users  for  whom  the  book  is  intended. 

The  table  of  contents  sent  to  the  publisher  ultimately  contained  30  articles  of  various  lengths.  The  publisher’s 
review  of  the  text  when  adjusted  to  a uniform  typographical  format  indicated  twice  the  number  of  pages  that  had  been 
estimated.  At  this  point,  retention  of  items  was  based  on  several  criteria:  (1)  difficulty  of  access;  (2)  older  items;  (3) 

"core"  of  the  literature;  and  (4)  published  in  the  United  States  (since  the  primary  users  would  be  in  North  America).  This 
wrenching  task  left  17  of  the  30  articles. 

The  conclusion,  based  on  the  experience  of  the  selection  process,  is  that  each  educational  technologist  has 
his/her  own  list  of  "classic"  articles  from  the  professional  literature.  This  idiosyncratic  opinion  is  based  on  such  factors 
as  the  university  in  which  graduate  study  was  done,  courses  taught  by  the  individual  and  the  satisfaction  received  from  a 
specific  publication  at  one  time  or  another  during  one’s  professional  career.  The  final  decision  as  to  what  to  include  is  a 
mix  of  the  authors'  personal  "favorites,"  the  "votes"  of  professional  colleagues  and  the  input  of  students  who  will  be  the 
readers  of  this  volume. 

Organization  of  the  Book 

There  is  no  one  conceptual  organization  of  educational  technology  that  is  acceptable  to  all  professionals.  It 
seems  that  each  person  views  the  field  from  a personal  perspective  based  on  the  individual’s  experience  with  those  aspects 
of  the  field  that  fall  within  the  scope  of  one's  daily  work.  When  it  comes  to  organizing  the  "classic"  works,  one  scheme 
would  appear  to  be  almost  as  logical  as  the  next.  The  authors  have  chosen  to  use  the  following  approach: 

1 . Definition  and  Conceptual  Background 

1 .2  The  field  and  its  definition 

1 .2  Theory  and  rationale 

2.  Design  and  Development  Functions 

2. 1 Design  and  development 

2.2  Evaluation 

3.  Delivery  Options 

3.1  Media 

3.2  Methods  and  techniques 

4.  The  Profession 

The  table  of  contents  is  appended  to  this  paper. 

A Final  Comment 

It  is  apparent  that  these  are  "dated"  articles.  Most  of  them  come  from  the  three  decades  following  the  end  of 
WorldWar  EL  Each  article  is  representative  of  the  author’s  contributions  to  the  field  of  educational  technology.  There  are 
ten  times  the  number  of  articles  contained  in  this  volume  that  constitute  the  core  literature  of  the  field.  Some  are  within 
the  range  of  dates  used  here;  many  more  are  published  later.  The  contents  of  this  book  are  "classics"  as  noted  by  many 
practitioners  in  the  field.  There  probably  would  not  be  100%  agreement  among  all  the  individuals  who  participated  in  the 
nomination  process.  In  the  book  readers  will  find  an  expanded  bibliography  of  many  works  that  have  helped  to 
establish  the  field  as  a profession. 
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Abstract 


Although  much  of  the  pedagogical  literature  suggests  otherwise,  not  all  students  find  case-based  instruction  interesting 
and  valuable  or  are  confident  learning  from  this  method.  In  this  exploratory  study,  we  examined  how  students  responded 
to  case-based  instruction  by  exploring  similarities  and  differences  among  nine  students'  experiences  in  a case-based 
biochemistry  course.  Nine  first-year  veterinary  students  were  interviewed  three  times  during  the  semester  to  explore  their 
initial  and  changing  responses  to  case-based  instruction.  Two  general  patterns  of  responses  emerged  during  our  analysis. 
These  patterns  revolve  around  the  value  that  students  assigned  to  case-based  instruction  and  describe  the  extent  to  which 
they  felt  motivationally  challenged  or  frustrated  by  the  case  method.  Implications  for  the  development  and  use  of  case 
studies  in  professional  education  programs  are  discussed. 

Although  case-based  instruction  has  been  accepted  as  an  effective  teaching  method  in  business  and  law  schools  for 
over  a century,  very  little  work  has  been  done  which  carefully  examines  how  individual  learners  respond  to  case-based 
instruction  (Knirk,  1991).  The  general  implication  in  the  literature  is  that  students  find  cases  motivating  (e.g.,  Shulman, 
1992),  yet  a few  educators  have  argued  that  case-based  instruction  might  not  "work"  for  all  learners  (e.g.,  Cossom,  1991). 
Given  the  fervor  with  which  case-based  instruction  is  currently  being  advocated  in  professional  education  (Shulman, 

1992),  it  is  important  to  understand  how  this  instructional  method  affects  the  persons  most  directly  involved  in  it.  What 
are  students’  perceptions  of  case-based  instruction?  How  interesting  and  valuable  do  students  find  this  method;  how 
confident  do  they  feel  learning  from  this  approach?  What  makes  this  type  of  learning  more  or  less  challenging  than  other 

instructional  methods? 

In  general,  case-based  instruction  is  a teaching  method  that  requires  students  to  actively  participate  in  real  or 
hypothetical  problem  situations  reflecting  the  kinds  of  experiences  naturally  encountered  in  the  discipline  under  study. 
Although  there  are  many  varieties  in  both  form  and  style,  case-based  instruction  tends  to  support  a focus  on  professional 
education  as  a process,  not  a product.  As  such,  it  is  believed  to  develop  practitioners  who  can  make  sense  of  problems 
that  are  not  always  straightforward  or  clear-cut. 

It  is  not  unusual  for  those  who  advocate  the  use  of  case-based  instruction  to  assume  that  students  will  be  motivated 
to  deepen  their  understanding  when  confronted  with  authentic  problems  in  realistic  situations  (Blumenfeld,  Soloway, 
Marx,  Krajcik,  Guzdial,  and  Palincsar,  1991).  Unfortunately,  not  all  students  are  adequately  prepared  to  direct  their  own 
learning  in  a case-based  environment.  Cossom  (1991)  stated  that,  "Clearly  (case-based  instruction)  is  not  a 
teaching/learning  method  that  appeals  to  all  students  nor  is  it  one  that  draws  neutral  responses  (p.  151).  Providing 
students  with  opportunities  to  integrate  their  knowledge  through  case  studies  may  not  be  effective  if  they  lack  the  skills 
or  motivation  needed  to  regulate  their  learning.  It  is  important  for  case  instructors  to  be  aware  of  students  responses  to 
the  case  method  and  to  provide  support  for  those  who  are  unprepared,  intimidated,  or  reluctant  to  engage  in  these 
unfamiliar  learning  tasks. 

This  study  departs  from  traditional  summative/evaluative  or  media/method  comparison  studies  m that  it  describes 
case-based  instruction  from  the  participants'  point  of  view  and  describes  those  aspects  that  learners  found  most  valuable 
and/or  frustrating  about  the  case  method.  By  examining  a variety  of  students'  responses  we  hoped  to  identify 
instructional  conditions,  learner  characteristics,  and/or  learning  strategies  that  facilitated  or  limited  students'  responses  to 
this  method.  Ultimately  our  goal  was  to  provide  educators  with  information  about  how  to  design/utilize  the  approach  so 
that  benefits  of  case-based  instruction  might  be  maximized  for  all  learners  expected  to  learn  from  it.  Thus,  the  questions 
guiding  data  collection  were 

(1)  How  do  students  respond  to  case-based  instruction?  Which  aspects  do  they  find  interesting,  valuable,  and 
worthwhile?  Which  aspects  are  difficult  or  frustrating? 

(2)  How  do  students'  responses  change  as  they  gain  experience  with  the  case  method?  What  happens  to  their 
interest,  value,  and  confidence  for  learning  from  this  method  during  a semester  long  course?  What  happens  to 
frustration? 


Methods 

This  study  used  qualitative  methodology  to  explore  the  perspectives  of  a diverse  group  of  veterinary  students  who 
were  recently  introduced  to  case-based  instruction.  Semi-structured  interviews,  conducted  with  nine  first-year  students, 
constituted  the  primary  data  source  and  were  supported  by  additional  data  gathered  from  classroom  observations,  students' 
written  case  analyses,  informal  teacher  interviews,  and  student  course  evaluations.  Qualitative  analysis  methods  were 
used  to  search  interview  data  for  patterns  of  responses  to  case-based  instruction.  Comparisons  were  then  made  across 
time  to  assess  changes  in  students'  responses  over  the  semester. 
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The  Site 


A professional  school  of  veterinary  medicine,  located  at  a large  midwestem  university,  \s  one  of  only  27,  nationwide, 
that  grants  the  degree  of  Doctor  of  Veterinary  Medicine  (DVM).  This  particular  school  follows  the  traditional  model  of 
veterinary  education,  which  according  to  Turnwald,  Bull,  and  Seeler  (1993),  "is  based  on  the  concept  that  the  primary 
purpose  of  education  is  transmittal  of  knowledge  and  skills"  (p.  38).  Didactic  instruction  is  provided  in  the  areas  of 
anatomy,  pathobiology,  physiology,  and  pharmacology,  and  tends  to  rely  on  "fact-laden  lectures,  assigned  readings  drills, 
quizzes,  rote  memorization,  and  examinations"  (p.  38).  Laboratory  experiences  are  included  in  the  first  three  years,  yet 
are  primarily  directed  toward  transmitting  information  to  the  students.  In  their  fourth  year,  students  assume 
responsibilities  of  "junior  practitioners"  in  actual  clinical  and  lab  settings. 

The  4-year  program  typically  limits  enrollment  to  60  students  per  class.  According  to  the  school  catalog,  "each 
prospective  student  is  required  to  complete  a prescribed  preprofessional  curriculum  for  two  or  more  collegiate  years  before 
admission  to  the  school."  Academic  performance  (e.g.,  overall  GPA,  required  courses  GPA,  overall  academic 
performance,  and  GRE  score)  accounts  for  50%  of  the  admissions  criteria,  and  non-academic  activities  (interview,  work 
experience,  extracurricular  activities,  veterinary  and  non -veterinary  animal  experience,  application  and  essay  quality,  and 
references)  constitute  the  other  50%.  Due  to  the  equal  weight  assigned  to  academic  and  non-academic  criteria,  the  student 
population  is  perceived  to  be  fairly  heterogeneous. 

The  Course 

A required  freshman  course.  Systemic  Physiology  II  (Biochemistry),  provided  the  context  for  this  study.  The  course 
is  described  in  the  syllabus  as  "an  introduction  to  biomedical  principles  and  their  application  to  veterinary  medicine." 

The  lab  portion  of  the  course  focused  on  the  application  of  principles  learned  in  lecture  to  hypothetical  patients  who 
mimicked  real-life  disorders.  At  the  time  of  the  study  the  same  instructor  taught  two  sections  of  the  lab.  Although 
students  were  assigned  to  one  lab,  they  typically  attended  the  one  that  was  most  convenient  during  any  specific  week. 
Therefore,  the  number  of  students  in  each  lab  section  fluctuated  between  15  and  45  students. 

Systemic  Physiology  II  was  designed  so  that  students  received  approximately  2 hours  of  both  lab  and  lecture  each 
week  throughout  the  16-week  semester,  yet  grades  were  weighted  such  that  lab  performance  accounted  for  only  27%  of 
the  total  course  grade  (a  common  practice  in  lecture-lab  courses  reflecting  the  perceived  "density"  of  their  respective 
information  loads).  Biochemical  case  studies  were  used  as  the  primary  instructional  method  and  as  an  evaluation  tool  in 
both  lab  sections  of  the  course.  Grades  assigned  in  lecture  were  based  on  3 objective- type  exams;  lab  grades  were  based 
on  a term  paper,  a group  case  presentation,  and  an  individual  case  analysis.  The  case  analyses  completed  during  lab 
meetings  were  not  graded  but  were  designed  to  give  students  practice  applying  biochemistry  principles  to  realistic 
problems  typically  encountered  in  professional  practice. 

Case  study  presentations  followed  a fairly  typical  pattern.  Students  were  presented  with  a limited  description  of  an 
animal  in  distress.  Some  of  the  animal's  symptoms  were  described,  accompanied  by  appropriate  illustrations  and 
diagrams,  and  then  students  were  asked  to  analyze  the  patient's  condition  and  to  make  tentative  diagnoses  and 
recommendations  for  action.  As  an  example,  one  of  the  cases  is  included  here: 

Signalment:  A German  Shepherd  dog,  castrated  male,  7 years  of  age. 

History:  This  dog  has  been  a family  pet  since  he  was  6 weeks  of  age.  He  received  a proper  series  of  "puppy 
shots"  and  has  been  vaccinated  every  year  for  rabies  and  DHLPP  (Distemper,  Hepatitis,  Leptospirosis, 
Parainfluenza,  and  Parvovirus).  He  receives  heartworm  prevention  through  the  spring,  summer  and  fall  of  each 
year,  and  has  tested  negative  for  heartworm  disease  each  year.  Normally  very  active,  he  has  been  lethargic  for 
the  last  7 to  10  days.  He  doesn't  want  to  run  and  play  like  he  used  to,  and  he  sleeps  a lot  more.  His  appetite  is 
decreased  a little,  and  he  threw  up  once  last  week.  The  family  has  noticed  that  he  coughs  in  the  morning  when 
he  gets  up  and  coughs  after  a nap. 

Physical:  Weight  is  87  pounds,  slightly  overweight.  Heart  rate  is  190,  temperature  is  101.4,  and  respiration  is 
20.  Appears  bright,  alert,  and  responsive  and  is  calm.  Mucous  membranes  are  more  pale  than  normal,  teeth 
have  some  tartar,  and  there  is  early  gingivitis.  Ears,  eyes,  and  throat  are  normal.  A lipoma  (diagnosed  last  year) 
is  present  over  the  left  thoracic  wall.  Thoracic  auscultation  reveals  harsh  lung  sounds,  and  a grade  3 of  0 
mum  r is  noted  in  the  fifth  intercostal  space  on  the  left  side,  vbdominal  palpation  is  normal.  Musculoskeletal 
system  is  normal  except  for  a slight  degree  of  muscle  wasting  in  the  rear  legs.  Nervous  system  appears  normal. 
Lab  results:  See  the  Complete  Blood  Count,  Chemistry  Profile,  and  Urinalysis. 
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Following  a standard  veterinary  analysis  procedure  called  "SOAP,"  students  analyzed  each  case  by  indicating  their 
subjective  (S)  and  objective  (O)  evaluations  of  the  animal,  their  assessment  (A)  of  the  animal's  condition,  and  their 
recommended  plan  (P)  for  action.  This  analysis  procedure  had  been  introduced  during  the  previous  semester  but  received 
increased  emphasis  in  the  biochemistry  lab.  Students  used  this  procedure  to  complete  their  case  analyses  throughout  the 
semester  yet  were  encouraged  to  streamline  the  process  as  the  semester  continued  so  that  it  required  less  time.  Students 
typically  worked  in  groups  to  complete  their  SOAPs  and  were  encouraged  to  ask  questions,  to  check  available  resources, 
and  to  consult  with  the  instructor  before  making  preliminary  diagnoses. 

All  case  investigations  concluded  with  a large  group  discussion  in  which  student  recommendations  were  considered  in 
light  of  the  available  lab  and  clinical  data.  A final  diagnosis  was  determined  after  a number  of  likely  diagnoses  were 
identified  and  rank-ordered  from  most  to  least  probable.  A discussion  of  the  biochemical  mechanism(s)  of  the  disease,  as 
well  as  the  effect  of  recommended  treatments  on  the  biochemistry  of  the  disorder,  served  to  link  case  specifics  to  basic 
biochemical  principles.  Although  the  teacher  indicated  why  the  selected  diagnosis  was  most  probable,  she  continually 
stressed  the  need  to  leave  many  possibilities  open,  even  those  not  initially  considered  to  be  very  likely. 


Participants 

Students.  Sixty-one  first-year  veterinary  students  were  enrolled  in  the  biochemistry  lab  during  the  study.  Sixty-six 
percent  of  these  students  were  female  (n  = 38),  and  ages  ranged  from  20  to  40  years  (M  = 24.22,  SD  = 3.99).  Although 
the  majority  of  students 

(n  = 36)  had  completed  a bachelor's  degree,  levels  of  education  ranged  from  two  years  of  post-secondary  education  (n  = 15) 
to  a master's  degree.  Most  students  had  backgrounds  in  either  biological  science  or  agriculture. 

In  order  to  capture  a wide  range  of  responses  to  case-based  instruction,  interview  participants  were  purposively 
chosen  to  represent  different  ages  and  genders,  and  to  include  a range  of  educational  and  work-related  experiences.  The 
final  interview  sample  included  7 female  and  2 male  students  ranging  in  age  from  21-32  years;  in  years  of  related 
veterinary  experience  from  0 - 13;  in  number  of  previous  related  courses  from  0 - 3;  and  with  GPAs  from  2.6  - 4.0. 

Table  1 presents  specific  demographic  information  for  the  nine  interviewees,  arranged  alphabetically  by  participants 
pseudonyms. 

Table  1. 

Demographic  Information  for  Interview  Participants 


Student 

Gender 

Age 

Yrs  of 
school 
(post 
HS) 

# of  previous 
biochemistry 
courses 

GPA 

Related  experience 

Chrissy 

F 

22 

4 

1 

3.53 

BS-Eng/Chem  double  major 

Deena 

F 

22 

4 

2 

2.71 

BS-Animal  Bioscience;  Pre-Vet 

George 

M 

21 

2 

0 

2.60 

Pre-Vet;  swine,  cattle  experience 

Mallory 

F 

23 

4 

3 

3.61 

Pre-Vet;  worked  in  Animal 
Control 

Marci 

F 

22 

3 

1 

4.00 

BS-Biology;  worked  4 yrs  in 
small  animal  clinic 

Ronald 

M 

23 

4 

0 

3.06 

BS-Biology;  raises  snakes 

Roslyn 

F 

32 

4 

1 

2.80 

13  yrs  in  vet  clinic;  3.5  yrs  in 
small  animal  ICU 

Sharon 

F 

24 

4 

2 

2.94 

BS-Animal  Science;  5 yrs  with 
small  animal  vet;  2 yrs  equine 
emergency  referral  service 

Winnie 

F 

26 

5 

0 

2.52 

BS-Biolopy;  dressage  instructor 

The  teacher.  At  the  time  of  the  study,  Eileen  Morrison  (a  pseudonym)  was  a 34-year-old  practicing  veterinarian,  as 
well  as  a graduate  student  in  the  School  of  Education.  Besides  working  half-time  at  a small  animal  clinic,  Eileen  also 
had  an  assistantship  that  required  her  to  teach  the  biochemistry  lab.  Of  the  other  eight  core  teachers  in  the  freshman 
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curriculum,  six  were  DVMs  and  held  advanced  degrees  (PhDs)  in  their  sciences;  two  had  advanced  degrees  only.  Of  these 
eight  core  teachers,  two  had  private  practice  experience. 

Eileen  had  been  teaching  this  particular  lab  for  two  years,  gradually  increasing  her  use  of  case  studies  until  they  had 
become  the  primary  instructional  method.  According  to  previous  course  evaluations,  students  liked  the  case  approach, 
typically  rating  the  lab  as  a 9.0  on  a 10-point  scale.  In  addition,  students  indicated  that  they  liked  Eileen  as  a teacher, 
with  average  semester  ratings  between  9.3  - 9.5.  Unsolicited  comments  from  .students  indicated  that  they  liked  her 
relaxed  teaching  style,  personable  manner,  and  sense  of  humor.  The  fact  that  Eileen  was  a practicing  veterinarian  seemed 
to  increase  both  her  credibility  and  approachability.  Eileen's  perception  was  that  students  trusted  her  to  present  practical 
and  relevant  information;  students  saw  her  knowledge  as  being  rooted  in  practice. 

Procedures 

Three  times  during  the  semester,  all  61  students  were  asked  to  complete  individual  written  case  analyses.  After  each 
of  these  cases,  semi-structured  interviews  were  conducted  with  the  9 selected  students  to  explore  their  responses  to  the 
case  method.  The  first  set  of  interviews  was  conducted  during  the  third  week  of  the  semester.  Second  and  third 
interviews  occurred  approximately  midway  through  the  semester  (week  8)  and  again  at  the  end  of  the  semester  (week  15). 
Specifically,  interviews  included  questions  related  to  students'  perceptions  of  their  interest/enjoyment  ("How  interesting  is 
this  instruction  to  you?";  "How  do  you  like  this  type  of  instruction?"),  value  ("How  valuable  is  this  approach  to  you?”), 
and  efficacy  ("How  confident  are  you  learning  from  this  method?"). 

These  27  interviews  constituted  our  primary  data  source  and  were  supported  by  secondary  sources  in  the  form  of 
students'  written  case  analyses,  classroom  observations,  teacher  case  documents,  informal  teacher  interviews,  and  course 
evaluations.  The  use  of  multiple  data  sources  and  methods  allowed  us  to  triangulate  analysis  efforts,  thus  reducing 
potential  subjectivity.  In  addition,  member  checks  were  secured  from  the  teacher  and  interviewees  throughout  the  research 
process. 

Data  Analysis 

To  answer  our  research  question  of  how  students  responded  to  case-based  instruction,  our  data  analysis  began  with  a 
search  for  patterns  of  responses  within  each  participant's  responses  (within-case)  and  then  across  all  learners  (cross-case) 
using  a constant  comparative  method  (Glaser  & Strauss,  1967).  By  examining  within-  and  cross-case  fluctuations  over 
the  semester,  we  were  able  to  describe  students'  changing  responses. 

Thus,  the  analysis  process  began  with  a search  for  students'  positive  and  negative  comments  across  all  interviews  and 
progressed  to  identifying  similarities  and  differences  among  comments.  For  example,  as  we  first  began  to  transcribe 
interviews,  we  noted  instances  where  students  expressed  enjoyment  or  frustration  related  to  the  case  method  in  general  or 
to  the  specific  case  they  had  analyzed  We  highlighted  the  reasons  students  gave  regarding  feelings  of  frustration  or 
enjoyment  and  constructed  tentative  matrices  that  outlined  similarities,  as  well  as  differences,  among  students’  responses. 
As  we  analyzed  subsequent  interview  comments  we  continued  to  modify  our  original  matrices — deleting,  adjusting,  or 
adding  categories  of  responses  to  reflect  emerging  themes. 

The  example  below  illustrates  our  analysis  approach.  In  the  first  interview,  students  were  asked,  "How  do  you  feel 
about  using  cases?"  We  present  Ronald’s  response  along  with  the  first  author’s  tentative  codings  (in  parentheses)  about 
the  meaning  of  his  response.  Ronald’s  reasons  lor  valuing  (or  devaluing)  the  case  approach  are  underlined. 

I enjoy  doing  them  (positive;  task  value?).  They  do  cause  frustration  (negative)  because  right  now  I really  don’t 
know  what  I’m  doing.  I don’t  have  a lot  of  background  (external  factors;  task  difficulty;  saving  face?).  I enjoy 
doing  it  (positive);  it’s  a change  of  pace.  Right  now  there’s  stress  (negative)  but  hopefully  that  will  change  with 
the  years  (positive)  as  I become  more  comfortable  with  them  (not  confident  now,  but  expects  to  improve)  and 
get  a wider  background.  But  now  they're  causing  stress  (negative)  because  there's,  well,  I have  no  idea  what  it 
could  possibly  be — I only  know  2 diseases  and  that’s  all  (excuses?  self-protection?). 

Based  on  this  small  interview  excerpt,  we  identified  a number  of  possible  themes:  value  due  to  enjoyment  and 
change  of  pace;  frustration  due  to  a lack  of  knowledge  and  an  uncertainty  of  what  one  was  supposed  to  be  doing;  poor 
performance  due  to  task  difficulty  and  lack  of  background  experience;  expectation  that  ability  and  confidence  would 
improve. 
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After  completing  this  level  of  coding  for  Ronald's  comments,  we  completed  similar  codings  for  the  other  eight 
students  interviewed.  By  looking  at  each  student's  response  to  this  question,  we  saw  similar,  as  well  as  additional  or 
opposing  themes.  By  continuing  to  look  for  similarities  and  differences  across  students'  responses,  we  gradually  clarified 
and  refined  our  codes  to  reflect  salient  themes  across  individuals,  while  still  noting  contrasting  features  and  contextual 
circumstances  surrounding  positive  and  negative  responses. 

As  themes  continued  to  evolve,  we  began  attending  to  changes  in  students’  responses  during  the  semester.  Whereas 
some  students  became  more  frustrated  over  time,  others  became  more  interested  and  more  motivated.  By  paying  close 
attention  to  the  condition  under  which  students'  responses  changed,  we  noted  different  patterns  of  responses  among 
students  who  remained  motivated  and  those  who  became  frustrated.  For  example,  students  began  to  express  more 
confidence  as  their  focus  changed  from  learning  facts  to  learning  the  case  approach  (product  vs.  process  goals).  On  the 
other  hand,  students  voiced  more  frustration  as  they  encountered  more  difficult  cases,  as  outside  pressures  mounted,  or  as 
the  novelty  wore  off.  These  critical  changes  in  responses  helped  us  identify  conditions  related  to  students'  facilitative  or 
limiting  responses,  taking  us  past  isolated  themes  and  categories  to  the  relationships  among  them. 


Results 

In  this  study  we  asked  questions  about  how  students  responded,  initially  and  over  time,  to  case-based  instruction. 
Two  general  patterns  of  responses  emerged  during  our  analysis  and  interpretation  of  the  data  collected  in  this  study. 

These  patterns  revolve  around  the  value  that  students  assigned  to  case-based  instruction  and  describe  the  extent  to  which 
they  felt  either  challenged  or  frustrated  by  the  case  method.  As  suggested  by  motivational  research  (e.g.,  Malone  & 
Lepper,  1987),  feeling  challenged  is  thought  to  be  a facilitative  response  whereas  feeling  frustrated  is  considered  a 
limiting  response.  Figure  1 illustrates  our  conception  of  how  levels  of  perceived  value  related  to  students'  responses  to 
case  learning  in  this  study.  Perceived  value,  as  defined  here,  includes  students’  levels  of  interest  in  the  use  of  cases  in  the 
biochemistry  lab  course,  the  perceived  relevance  of  the  case  method  to  their  current  coursework  and  future  career  goals, 
and  levels  of  confidence  for  both  successful  performance  and  ability  to  learn  from  this  method.  Prior  to  presenting  a 
description  of  the  two  general  response  patterns,  we  use  excerpts  from  students'  interviews  to  illustrate  how  each 
component  of  perceived  value  (interest,  relevance,  and  confidence)  shaped  students'  responses  to  this  case-based  course  in 
facilitative  or  limiting  ways. 
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Figure  1. 


Relationship  between  levels  of  perceived  value  and  students'  responses  to 
case-based  instruction. 


Component  On? — Initial  and  Changing  Interest  in  CaseJtased  Instruction 

Initial  interest.  In  describing  their  initial  reactions  to  the  use  of  cases  in  the  biochemistry  lab,  eight  of  the  nine 
interviewed  students  indicated  that  they  thought  that  cases  would  make  the  class  more  interesting  and  more  fun.  In  the 
first  interview  Winnie  explained,  "Of  course  you  are  more  interested  in  the  case  because  it  relates  to  medicine  and  to  what 
you  want  to  do  later  in  life.  Biochemistry,  or  physiology,  or  anatomy  is  very  dry  and  it's  not  alive.  But  working  with 
diseases — it  kind  of  gets  you  more  interested."  Mallory  indicated,  "Biochemistry  lab  is  the  most  interesting  course  that 
we  have." 

Four  students  likened  cases  to  a game  or  puzzle  and  mentioned  the  challenging,  enjoyable  aspects  of  cases.  Mallory 
stated,  "It's  more  of  a challenge;  it's  like  playing  a game  to  see  if  you  can  win.  It's  fun."  Six  students  described  how 
their  motivation  had  increased  and  one  student,  Marci,  even  mentioned  how  cases  had  affected  her  efforts  outside  of  class: 
"It  motivates  me  to  do  a lot  of  extra  reading  because  a lot  of  stuff  you  don't  get  just  by  going  to  classes."  For  the  most 
part,  students  indicated  that  case-based  instruction  was  more  interesting  than  their  other  classes  and  provided  a nice  change 
of  pace.  Mallory  contrasted  case-based  instruction  with  her  other  courses,  "I  like  them  (cases).  I mean  it's  not  boring. 

It's  not  ’memorize  these  facts  and  spit  them  back  to  me.'"  Only  Deena  qualified  her  comments  stating,  "They're  good, 
but  only  to  a certain  degree." 

Changing  interest.  By  the  end  of  the  course,  Chrissy,  Deena,  and  Ronald  mentioned  that  casework  was  becoming 
tedious  and  that  their  motivation  had  decreased  due  to  other  pressures.  Chrissy  stated,  "Sometimes  it  got  to  be  a really 
long  afternoon  when  we're  going  over  the  same  things."  Even  though  these  students  were  "burned  out"  by  the  end  of  the 
semester,  most  of  their  frustration  seemed  related  to  outside  sources  (e.g.,  other  tests,  deadlines),  rather  than  to  the  course. 
Ronald  explained,  "Right  now  things  are  swamped,  end  of  the  year.  I'm  just  burned  out  I guess."  Course  evaluations 
support  the  conclusion  that  most  students  still  enjoyed  the  course  at  the  end  of  the  semester.  On  a scale  from  1-10  the 
average  course  rating  was  8.9.  Unsolicited  student  comments  on  the  evaluation  form  included  enjoying  the  case-study 
approach  and  having  fun  in  class:  "Truly  enjoyed  labs;  they  made  my  others  relevant."  "I  greatly  enjoyed  lab!!!  It  was 
the  only  class  in  our  curriculum  which  makes  you  think  logically  about  cases  you  will  see  as  a clinician." 

Component  Two — Initial  and  Changing  Relevance  for  Case-Based  Instruction 

Initial  relevance.  All  nine  students  claimed  that  cases  were  "real-life"  and  had  some  practical  benefits.  Chrissy, 
Deena,  and  Ronald  noted  that  cases  would  help  them  remember  more,  yet  still  judged  that  this  would  probably  not  affect 
other  course  work  or  career  goals.  Chrissy  stated,  "I'm  planning  to  be  a non-practicing  veterinarian  so  I can't  really  predict 
what  role,  if  any,  these  cases  will  have."  Deena  explained,  "It's  going  to  help  me  learn  things  better,  but  I don't  think 
I'm  going  to  be  remembering  these  cases  when  I'm  working  in  the  real  world."  This  contrasts  with  five  other  students 
who  stated  that  the  case  method  was  very  valuable  to  their  future  careers,  as  well  as  to  other  coursework.  Marci  stated, 
"Cases  will  definitely  help  me  out  in  the  future.  They  will  help  me  be  a better  veterinarian."  Not  only  did  these  students 
value  the  practicality  of  case-based  instruction,  but  they  also  noted  some  global  benefits  such  as  learning  the  problem- 
solving approach  and  integrating  their  knowledge.  Sharon  stated,  "I’m  in  the  situation  where  I'm  trying  to  pull  together 
everything  that  I learned  in  biochemistry  as  an  undergrad  and  things  we’re  learning  in  physiology  and  anatomy  and  pull 
everything  together  through  biochem  lab.  The  case  studies  kind  of  integrate  it  and  you  see  where  everything  connects." 

Changing  relevance.  As  the  semester  went  on,  there  seemed  to  be  a shift  regarding  which  aspect  of  the  case  approach 
was  valued  most.  Students  who  initially  focused  on  practical  benefits  (change  of  pace,  ability  to  remember  more  facts) 
began  to  mention  more  overarching  benefits  (application  of  knowledge,  learning  the  problem-solving  approach).  The 
case  analysis  process,  rather  than  the  product,  seemed  to  take  on  increased  relevance.  George  stated,  "I  think  the  biggest 
thing  to  me  is  the  whole  process  of  thinking  through  the  different  diagnoses.  It's  just  a whole  mind  set  that  we're  getting 
into."  Ronald  mentioned  how  the  case  method  helped  him  organize  his  thoughts  and  decided,  as  did  Chrissy,  that  cases 
could  help  in  applying  knowledge  learned  in  other  courses. 

Component  Three — Initial  and  Changing  Confidence  for  Case-Based  Learning 

Initial  confidence.  All  of  the  students  expressed  some  concern  about  their  ability,  at  this  point  in  their  careers,  to 
diagnose  the  cases  they  were  given.  Marci  explained,  "I  was  a little  intimidated  because  I knew  I didn't  have  a lot  of 
knowledge  to  help  me  figure  out  what  was  going  on."  Chrissy  stated,  "Right  now  I’m  not  terribly  confident  in  my 
performance."  Students  used  words  such  as  scared,  frustrated,  nervous,  and  intimidated.  However,  everyone  but  Deena 
indicated  that  this  lack  of  knowledge  would  lead  to  greater  effort.  Marci  said,  "I  probably  put  more  effort  into 
understanding  what  we  learn  in  this  class  because  I know  it  will  definitely  be  useful." 
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Changing  confidence.  As  students  became  more  comfortable  with  the  problem-solving  approach,  as  their  knowledge 
base  increased,  and  as  their  experience  with  cases  increased,  they  appeared  more  confident  of  their  case  analyses. 

However,  students'  confidence  seemed  primarily  related  to  the  amount  of  prior  knowledge  and  previous  experiences  they 
had.  Ronald  indicated,  "If  I had  a broader  repertoire  of  possibilities,  I would  have  felt  more  confident."  Still,  Roslyn  and 
Marci  mentioned  being  motivated  by  this  lack  of  knowledge.  Roslyn  remarked,  "It's  like  a kid  with  a new  video  game!" 

As  the  semester  continued,  students  seemed  to  redefine  success  and  to  adjust  their  judgments  of  confidence  to  match. 
They  began  to  emphasize  "coming  close”  rather  than  naming  a specific  disease.  If  diagnoses  were  "in  the  ballpark" 
students  judged  their  work  to  be  successful.  Mallory  said,  "I  knew  this  and  this,  but  being  able  to  list  a specific 
problem,  no,  I don't  know  enough  diseases  to  write  anything  down.  But  it  comes  close." 

By  Time  3,  scared  and  nervous  feelings  were  no  longer  mentioned,  yet  Chrissy,  Deena,  and  Ronald  expressed 
fnistration  due  to  a lack  of  knowledge,  the  specific  case,  or  tediousness  of  the  work.  These  students  were  more  apt  to 
complain  about  other  responsibilities,  other  course  requirements,  and  external  factors  (time  and  length  of  lab)  that 
contributed  to  their  stress.  Deena  explained,  "I  got  frustrated  because  when  I have  so  many  other  things  to  be  doing,  I 
don't  want  to  do  it."  Marci,  Sharon,  and  Winnie  also  expressed  concern  about  a lack  of  knowledge,  yet  tended  not  to 
dwell  on  this.  Rather,  they  reminded  themselves  of  the  overall  value  of  casework.  Winnie  stated,  "Although  it's 
frustrating  if  I get  every  one  of  them  'wrong,'  I think  this  is  the  way  to  learn.  Sharon  and  Marci  both  indicated,  Well 
be  better  clinicians  because  of  it." 

Summary 

Students  in  this  course  were  observed  to  respond  differently  to  case-based  instruction  based  on  their  perceived  value 
for  the  case  method,  including  their  interest  in  the  use  of  cases,  the  perceived  relevance  of  cases  to  their  current  and  future 
work,  as  well  as  their  perceived  confidence  for  learning  from  this  approach.  Furthermore,  these  responses  appeared  to 
vary  as  interest,  relevance,  and  confidence  levels  changed.  Although  all  of  the  students  interviewed  in  this  study  found 
the  case  method  interesting  and  valuable  at  the  start  of  the  semester,  value  "levels"  were  not  equivalent  across  students. 
Whereas  some  students  simply  stated  that  cases  were  enjoyable  and  a nice  change  of  pace,  others  described  important 
connections  to  their  other  coursework  or  career  goals,  as  well  as  to  their  overall  motivation  and  confidence  for  learning. 

As  indicated  earlier,  students'  perceived  value  changed  during  the  semester,  both  by  level  and  by  type.  That  is  to  say, 
some  students  changed  in  terms  of  the  amount  of  value  they  assigned  to  case  learning  (with  some  assigning  more,  others 
assigning  less)  as  the  semester  progressed;  some  changed  their  reasons  for  valuing  the  method,  for  example  from  a focus 
on  the  enjoyment  aspects  to  one  on  perceived  relevance  to  future  career  goals.  These  varying  levels  and  types  of 
perceived  value  appeared  pervasive  in  influencing  students'  initial  and  subsequent  responses  to  case-based  instruction.  For 
example,  students  who  initially  placed  high  value  on  the  integrative  and  linking  functions  of  cases  (e.g.,  Marci,  Sharon, 
Winnie)  appeared  to  maintain  a high  value  for  the  case  method  throughout  the  semester.  Students  who  changed  their 
emphases  from  practical  and  immediate  benefits  to  more  long-term  benefits  (e.g.,  George,  Mallory,  Roslyn)  appeared  to 
increase  in  perceived  value  for  the  case  method.  Still,  a few  students  (e.g.,  Chrissy,  Deena,  Ronald)  who  initially  valued 
cases  because  they  were  interesting  and  "not  too  hard,"  decreased  in  perceived  value  as  the  novelty  wore  off  and  cases 
became  more  difficult. 

In  general,  students  who  valued  the  case  method  as  a useful  learning  tool  tended  to  emphasize  learning  the  case 
analysis  process  and  directed  their  efforts  toward  mastering  the  analysis  approach.  These  students  seemed  to  prefer  tasks 
which  were  new,  challenging,  or  difficult  so  "they  could  learn  from  them."  Students  who  did  not  perceive  the  relevance 
of  the  method  to  their  current  or  future  work,  and/or  who  lost  interest  in  the  method  seemed  more  interested  in  learning 
specific  biochemical  facts  and  did  not  seem  to  enjoy  the  challenge  provided  by  new  and/or  difficult  cases.  Wigfield  (1994) 
suggested  that  when  learners  are  engaged  in  tasks  for  utilitarian  purposes  (e.g.,  to  complete  a task,  to  get  a grade)  they 
may  not  wish  to  be  as  challenged  as  those  who  hold  high  value  for  the  task.  If  a task  is  too  challenging,  such  learners 
may  begin  to  look  for  other  tasks  that  could  meet  their  utilitarian  needs  in  less  challenging  ways. 

In  summary,  students  who  described  case-based  instruction  as  being  relevant  to  their  current  and  future  work,  who 
expressed  high  interest  in  and  enjoyment  of  the  case  teaching  method,  and  who  were  confident  that  they  could  learn 
relevant  skills  and  information  in  this  manner,  appeared  motivationally  challenged  by  the  case  method.  In  contrast, 
students  who  expressed  concern  that  they  might  not  learn  all  the  biochemical  facts  they  needed  to  know,  who  were  not 
convinced  of  the  relevance  of  these  cases  to  their  other  work,  and  who  were  unsure  of  their  ability  to  perform 
successfully,  felt  frustrated  by  this  approach. 


Discussion 

The  results  of  this  exploratory  study  point  to  the  potential  role  that  perceived  value  (as  defined  by  levels  of  interest, 
relevance,,  and  confidence)  may  play  in  shaping  students'  responses  in  a case-based  course.  Although  our  results  must  be 
regarded  as  tentative,  given  the  small  number  of  participants,  they  suggest  important  areas  for  future  research.  Perceived 
value,  as  a motivational  component  of  learning,  has  been  described  in  the  literature  as  influencing  one's  willingness  to 
learn.  Wigfield  (1994)  suggested  that  learners’  "valuing  of  different  tasks  may  be  an  important  precursor  of  their 
willingness  to  devote  time  and  energy  needed  to  become  proficient  at  that  task"  (p.  121). 

Although  there  are  many  who  may  agree  with  Wassermann's  (1994)  claim  that  "case  method  teaching  can  be 
effectively  applied  in  virtually  every  subject  area,  at  most  educational  levels"  (p.  1 1),  case-based  instruction  was  not 
equally  beneficial/meaningful  for  all  learners  in  this  study.  By  examining  the  responses  of  a variety  of  learners,  we 
identified  aspects  of  case-based  instruction  that  were  most  valuable  and  most  frustrating  for  this  group  of  learners.  These 
results  provide  important  insights  regarding  how  to  design  and  structure  case-based  learning  environments  so  as  to 
increase  students'  valuing  of  both  the  case  analysis  approach  and  specific  case  analysis  tasks. 

Based  on  comments  from  these  nine  students,  we  suggest  specific  aspects  of  the  case  method  that  might  be  re- 
designed to  optimize  student  learning.  For  example,  when  instructors  are  preparing  to  implement  a case  approach,  they 
might  begin  by  considering  the  following  questions:  Do  students  understand  what  they  are  supposed  to  do  and  why  they 
are  doing  it?  Do  students  perceive  the  relevance/meaningfulness  of  the  cases/case  method  outside  of  the  classroom 
setting?  Are  the  cases  perceived  as  being  authentic  and  realistic?  Are  they  interesting  and  enjoyable?  Is  there  enough 
variety  to  keep  interest  and  motivation  high?  Do  students  believe  that  they  can  master  the  case  approach  with  a 
reasonable  amount  of  effort?  Are  there  opportunities  to  learn  from  mistakes  and  to  learn  from  and  with  others?  Are 
students  involved  in  the  design  and  selection  of  tasks?  Are  students  permitted/  encouraged  to  make  choices  regarding  how 
they  learn?  By  designing  case-based  learning  environments  with  these  questions/concems  in  mind,  students  may  be  better 
able  to  see  the  value  which  many  educators  believe  is  inherent  in  the  method.  Furthermore,  it  is  expected  that  by 
increasing  students'  value  for  the  case  method,  effective  learning  will  be  more  likely  to  occur  (Brown  & DeLoache, 

1978). 

Limitations  and  Directions  for  Future  Research 

In  addition  to  the  small  number  of  participants,  several  components  of  this  study  limit  its  comparability.  First,  this 
study  described  students'  responses  to  one  variation  of  the  "case  method."  Students'  responses  might  be  expected  to  vary 
with  the  specific  type  of  case/case  method  used.  It  would  be  useful  to  examine  how  differences  in  case  design  and 
purpose  influent  c students’  responses.  Second,  veterinary  students  may  not  be  representative  of  students  in  other 
disciplines  that  use  case  methods.  It  would  be  important  to  examine  the  responses  of  students  in  a variety  of  disciplines. 
Third,  in  this  study  we  observed  one  instructor's  use  of  case  studies  in  one  content  area  and  thus  cannot  separate  specific 
effects  due  to  the  instructor  from  those  due  to  the  method,  content,  or  participants.  Perhaps  if  this  instructor  used  a 
traditional  method,  students  would  still  haye  responded  as  they  did.  Or  perhaps,  this  group  of  students  would  have 
responded  similarly  to  this  and/or  other  instructional  methods  regardless  of  the  teacher.  Comparisons  among  students' 
responses  to  different  instructors  and  different  methods  used  by  the  same  instructor  might  clarify  this  picture. 

Conclusion 

This  study  examined  students’  responses  to  case-based  instruction  by  exploring  the  similarities  and  differences  among 
nine  students'  experiences.  Understanding  how  case-based  learning  is  experienced  by  the  participants  requires  in-depth 
interviews  with  a variety  of  students  to  gain  their  unique  perspectives.  It  is  our  hope  that,  having  heard  the  voices  of 
these  nine  learners  as  they  encountered  the  challenges  and  difficulties  of  an  unfamiliar  and  demanding  educational 
approach,  educators  will  be  in  a better  position  to  help  future  students  effectively  respond  to  similar  demands. 

One  important  implication  of  this  study  is  the  need  to  inquire  into  students’  vaiue  for  the  educational  tasks  and 
methods  they  encounter.  The  fact  that  students  responded  differently  to  the  same  instructional  method  highlights  the 
importance  of  attending  to  individual  students'  perceptions  of,  and  responses  to,  classroom  experiences.  Although  case- 
based  instruction  was  highly  motivational  and  engaging  for  some,  it  was  a difficult  and  frustrating  learning  experience 
for  others.  Rather  than  assume  that  case-based  instruction  automatically  works  for  all  learners,  it  is  important  for 
educators  to  assess  their  students’  initial  perceptions  and  skill  levels  and  then  modify  or  enhance  the  case  approach  to 
meet  their  needs.  By  being  aware  of  the  effects  that  perceived  value  may  have  on  students'  responses,  educators  may  be 
able  to  alter  or  eliminate  potential  difficulties  before  they  become  problematic.  As  Daloz  (1990)  reminds  us, 
"Understanding  the  precise  nature  of  the  demands  we  make  on  our  students  and  calibrating  our  expectations  to  their 
particular  strengths  and  weaknesses  is  a special  art"  (p.  89)  yet  this  is,  after  all,  what  all  good  teachers  are  expected  to  do. 
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It  is  hoped  that  the  results  of  this  study  will  help  case  designers  and  case  teachers  meet  the  challenge  of  supporting 
students  in  their  efforts  to  become  effective  case  learners. 
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Abstract 


The  use  of  computers  as  a tool  for  learning  has  traditionally  focused  on  individualized  methods  of  instruction  (Lumsdaine 
& Glaser,  1960;  Saettler,  1990).  Social  interaction,  however,  is  taking  an  increasingly  important  role  in  current  learning 
theories  and  instructional  prescriptions  including  computer-based  delivery  systems  (Bruffee,  1993;  Lave,  1988). 
Concurrent  with  these  recommendations,  global  computer  networks  have  emerged  bringing  new  forms  of  computer- 
mediated  social  interaction.  This  paper  discusses  an  ongoing  case  study  that  takes  advantage  of  Internet  technology  to 
' promote  learning  through  socially-mediated  interactions.  Based  upon  theories  of  situated  learning  and  cognitive 
apprenticeship  (Brown,  1989;  Collins,  1990),  students  engage  in  social  dialog  with  experts  through  Internet  technologies 
such  as  email,  email  redistribution  lists  (listservs)  and  the  World-Wide  Web. 


Background 

The  use  of  computers  as  a tool  for  learning  has  generally  focused  on  individualized  methods  of  instruction  (Lumsdaine  & 
Glaser,  1960;  Saettler,  1990).  Largely  influenced  by  behaviorism,  traditional  computer-based  training  reflects  a stimulus- 
response  approach.  Highly  structured  and  repetitive  interactions  delivering  immediate  feedback  are  still  standard  features 
within  many  computer-based  learning  environments.  Approaches  derived  from  the  cognitive  sciences  (such  as  Intelligent 
Tutoring  Systems)  have  also  featured  individualized  instruction  (Farquhar  & Orey,  in  press;  Sleeman  & Brown,  1982). 
With  renewed  influences  from  philosophy,  sociology  and  anthropology,  epistemologies  and  theories  such  as  social 
constructivism  and  situated  cognition  suggest  new  approaches  for  the  design  and  implementation  of  learning 
environments.  Concurrent  with  the  arrival  of  these  new  perspectives,  technologies  have  emerged  giving  us  new 
capability  to  engage  learners  in  structured  social  interaction. 

New  Perspectives 

Social  constructivism  posits  that  individual  constructions  of  knowledge  derive  from  interactions  with  the  social 
environment  (Cobern,  1993;  Tobin  & Tippins,  1994).  From  this  perspective,  knowledge  is  continually  constructed  and 
reconstructed  by  the  individual.  The  social  environment  provides  a set  of  experiences  from  which  the  individual  tests 
understanding  and  adopts  group  norms.  Viable,  learning  environments,  from  a social  constructivist  viewpoint,  structure 
social  interactions  to  support  the  development  of  personal  meaning. 

This  philosophical  position  is  consistent  with  situated  cognition,  a theory  that  describes  knowledge  as  a relationship 
between  cognitive  and  social  factors  (Brown,  1989;  Streibel,  1994).  The  primary  supposition  of  situated  cognition  is 
that  knowledge  exists  only  within  context.  According  to  this  theory,  an  understanding  of  knowledge  must  include  the 
artifacts,  activities,  participants,  and  interactions  within  a cultural  setting.  Orey  and  Nelson  (in  press)  suggest  that  the 
theoretical  position  of  situated  cognition  is  a cross-disciplinary  perspective  involving  ideas  from  both  psychology  and 
anthropology.  Lave  (1988),  an  anthropologist  studying  situated  cognition,  describes  the  everyday  use  of  knowledge  as 
"distributed  - stretched  over,  not  divided  among  - mind,  body,  activity  and  culturally  organized  settings. 


While  social  constructivism  and  situated  cognition  are  views  with  which  to  understand  and  describe  knowledge,  cognitive 
apprenticeship  is  an  amenable  methodology  for  the  acquisition  of  knowledge  (Collins,  1990).  The  cognitive 
apprenticeship  model  proposes  that  the  design  of  learning  environments  reconstruct  important  attributes  of  master- 
apprentice  relationships.  Among  other  attributes,  "apprenticeship  embeds  the  learning  ot  skills  and  knowledge  in  their 
social  and  functional  context"  (Collins,  1990,  p.454).  Authentic  tasks  and  meaningful  situations  are  required  to  bring 
novices  into  a culture  of  expert  practice  (Brown,  1989;  Vanderbilt,  1990).  Thus,  situated  learning  is  a recognized  and 
employable  feature  within  the  cognitive  apprenticeship  framework. 

The  cognitive  apprenticeship  model  makes  a number  of  recommendations  for  the  design  of  learning  environments.  In 
addition  to  providing  a meaningful  and  authentic  task,  the  model  calls  for  reflection,  articulation,  collaboration,  and 
multiple  practice.  Cognitive  apprenticeship  prescribes  the  implementation  of  these  features  through  structured  social 
interaction  within  a community  - a process  that  Lave  describes  as  "legitimate  peripheral  practice.  Until  the  recent 
development  of  the  global  network,  structured  social  interaction  has  not  been  an  instructional  approach  easily  supported 
by  computer  technologies.  The  ongoing  study  described  in  this  paper  has  examined  the  implementation  of  cognitive 
apprenticeship  strategies  through  the  use  of  recently-available  Internet  technologies. 
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Study  Context 

Over  a period  of  four  college  semesters  a variety  of  Internet  technologies  were  implemented  by  the  researchers  in  four 
separate  graduate-level  courses.  All  four  courses  were  related  to  the  topic  of  training  technologies,  yet  had  a different 
focus.  The  majority  of  the  participants  (75%;  n=58)  were  completing  introductory  courses  in  training  technologies, 
while  the  minority  were  taking  development  courses  in  either  computer-based  training  (15%;  n=l2)  or  video  production 
for  training  (10%;  n=8).  Upon  entering  the  courses,  few  subjects  had  prior  experience  with  Internet  technologies.  The 
fall  and  spring  classes  met  for  one  evening  per  week,  while  the  one  summer  course  met  twice  per  week. 

The  technologies  selected  to  support  the  strategies  included  personal  e-mail  among  participants,  e-mail  distribution  lists 
(listservs),  synchronous  "chat"  communications,  and  the  World-Wide  Web.  Consistent  with  the  cognitive  apprenticeship 
model,  the  strategies  implemented  were  characterized  by: 

Complex,  problem-based  scenarios  were  selected  in  order  to  situate  or  anchor  knowledge  within  a variety  of  contexts. 

Articulation  was  encouraged  through  open-ended  requests  to  propose,  posit,  or  recommend. 

Reflection  of  personal  experiences  was  encouraged  through  requests  to  defend  and  rationalize. 

Responses  to  scenarios  by  all  parties  were  most  often  "published”  or  made  available  for  all  in  a socially-mediated, 
consensus-building  environment. 

Novices  were  encouraged  to  collaborate. 

Data  was  collected  through  observations,  records  of  electronic  communications,  interviews  with  selected  subjects,  and 
self-reports.  This  study,  which  continues  through  the  present  semester,  intends  to  reveal  how  skill  development  can  be 
supported  or  impeded  by  the  strategies  and  technologies  selected. 

Procedure  with  Listserv  & Email 

The  first  technology  implemented  was  electronic  mail  (e-mail)  and  an  e-mail  redistribution  list  (in  this  case  known  as  a 
listserv).  All  students  were  given  an  e-mail  account  and  a brief  introduction  to  its  use.  A few  students  implemented  e- 
mail  capabilities  from  their  home  computer  through  either  the  University  system  or  by  subscribing  to  a commercial 
service. 

All  students  were  subscribed  to  a newly  assigned  listserv  managed  and  controlled  by  the  course  instructor  and  lead 
researcher.  Two  special  guests  were  regularly  subscribed  to  the  listserv  with  the  specific  responsibility  to  regularly 
respond  to  questions  and  comments  as  consulting  experts  in  the  field  of  training  technologies.  Brendan  McGinty,  now 
Vice  President  of  Development  & Operations  with  University  Communications,  has  20  years  as  a Computer-Based 
Training  and  Multimedia  Developer.  Carrie  Kotcho  serves  Bell  Atlantic  as  a Multimedia  Developer  and  has  10  years  of 
experience  producing  training  videos  and  multimedia  products.  While  introduced  as  "virtual  guest  speakers,”  the  intent 
behind  their  participation  was  to  include  them  as  expert  members  of  the  community,  or  "mentors,”  to  promote  a 
cognitive  apprenticeship  relationship  with  classmembers. 

Subscribers  to  the  list  were  encouraged  to  introduce  themselves  and  to  post  any  messages  that  they  felt  relevant  to  the 
topic  of  training  technologies.  The  two  smaller  classes  were  given  specific  assignments  to  introduce  themselves  to  the 
list. 

Assignments  that  involved  use  of  Internet  communications  were  characterized  by  an  ill-defined  problem,  based-upon  a real 
(or  realistic)  situation  lacking  in  sufficient  detail  to  render  an  immediate  solution.  Students  were  asked  to  propose  a 
solution  to  the  problem  with  the  experts  acting  as  either  resources  or  clients.  Figure  1 describes  one  of  the  given 
scenarios  where  Brendan  McGinty,  an  expert  in  the  implementation  and  application  of  the  NovaNET  CBT  system,  acted 
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as  a mentor.  In  most  cases,  teams  were  formed  (from  two  to  four  students)  to  worked  on  problems  with  the  request  that 
they  post  all  questions  and  responses  to  the  list. 


Results  from  Listserv  and  Email 

The  teams  collaborated  for  a period  of  typically  less  than  one  hour  before  generating  and  posing  questions  for  the  expert. 
While  the  request  was  made  to  post  all  questions  to  the  listserv,  a few  teams  sent  questions  directly  to  the  expert  through 
his  e-mail  address.  Reported  reasons  for  this  were  both  a misunderstanding  of  the  directions  as  well  as  inadvertent  or  an 
unintended  use  of  the  e-mail  system. 

Nearly  all  questions  posed  by  the  students  at  this  early  stage  of  the  assignment  were  lengthy  descriptions  of  the  problem. 
Most  of  the  postings  to  the  listserv  were  summaries  of  the  given  scenarios;  one  team  simply  copied  the  given  problem 
description  to  the  listserv  with  an  additional  question  at  the  end. 

With  minor  exceptions,  the  students  reported  that  the  problems  were  meaningful  and  relevant.  On  more  than  one 
occasion,  a team  admitted  to  intentionally  forming  some  assumptions  of  the  problem  around  their  own  work-related 
context  or  present  need.  Upon  consultation  with  the  instructor,  one  team  generated  their  own  problem  directly  from  their 
present  work  environment. 


Engineering  Services  Scenario; 

A company  providing  a broad  range  of  engineering  services  with  headquarters  in  Raleigh-Durham,  North  Carolina,  is 
dedicated  to  maintaining  a highly-educated  workforce  knowledgeable  in  current  technologies.  The  headquarters  provides 
offices  for  120  mechanical,  electrical,  and  computer-science  engineers.  An  additional  25  personnel  are  employed  in 
managerial  and  support  staff  roles.  Nearly  all  employees  have  a college  degree,  many  posses  graduate  degrees. 

The  company  boasts  its  support  for  the  personal  growth  of  its  employees.  Nearly  all  of  its  employees  use  computers  in 
their  daily  activities.  Computers  are  assigned  to  each  employee  and  a number  of  regular  workshops  are  offered  in  various 
software  applications.  Most  company  computers  are  high-end  systems  capable  of  funning  sophisticated  modeling 
programs  and  are  connected  to  various  networks  including  the  Internet. 

The  company  requires  that  each  employee  complete  40  hours  of  training  and  development  programs  each  year. 

Employees  select  programs  through  consultation  with  management.  The  majority  of  training  programs  provide 
employees  with  an  opportunity  to  increase  their  engineering  skills  in  a particular  area  or  to  broaden  their  skills  to  other 
areas.  Unfortunately,  the  present  training  facilities  are  overbooked  and  understaffed.  Many  employees  are  waiting  six 
months  or  more  to  schedule  some  of  the  more  popular  courses. 

This  company  is  seeking  to  expand  its  training  programs,  but  is  looking  at  cost-effective  alternatives  to  simply 
providing  additional  training  staff  and  space.  So  far,  the  company  has  not  made  use  of  CBT  as  a delivery  medium. 

Would  you  advise  the  company  to  provide  CBT?  And,  what  services,  if  any,  from  NovaNET  would  you  recommend  that 

they  provide? 

Figure  1 . Example  problem  for  listserv  discussion. 

Responses  from  the  experts  to  the  initial  set  of  questions  occurred  anywhere  from  one  to  five  days  later.  The  responses 
were  most  often  lengthy  discussions  presenting  several  points.  Since  all  responses  to  the  questions  were  posted  to  the 
listserv,  the  experts  would  often  refer  to  points  made  in  their  previous  comments. 

A couple  of  teams  were  able  to  develop  their  solution  to  the  problem  after  only  one  interchange  with  the  expert.  Other 
teams,  however,  asked  follow-up  questions  approximately  one  week  later.  On  very  few  occasions  did  the  dialog  continue 
through  three  rounds  of  questions  and  answers.  The  lack  of  follow-up  dialog  through  more  than  a single  round  of 
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questioning  was  consistent  in  most  dialogs  on  the  listserv  despite  the  researchers'  intent,  through  on-line  encouragement, 
to  "keep  the  questions  coming." 

In  class,  students  reported  a respect  for  the  experts  and  the  information  provided  for  them  through  the  listserv. 
Additionally,  students  reported  an  interest  in  continuing  a dialog  with  the  experts  with  a few  comments  suggesting  that 
the  listserv  dialog  was  "very  short." 

Procedure  with  Synchronous  Chat 

One  small  class  (n=8)  during  the  summer  of  1995  was  engaged  in  synchronous  chat  sessions  with  an  expert  centered  on 
solving  an  ill-defined  problem.  In  most  cases,  students  were  asked  to  propose  a training  video  as  the  solution  to  a 
particular  training  problem.  The  chat  was  arranged  through  a commercial  on-line  service  (America  Online)  due  the 
availability  of  such  service  by  the  expert.  The  interface  for  the  synchronous  chat  allows  messages  of  no  more  than  two 
lines  to  be  immediately  posted  within  a scrolling  window. 

Pairs  of  students  were  formed  to  respond  to  the  expert’s  questions.  Each  pair  was  advised  to  collaborate  on  the  problem 
prior  to  meeting  the  expert  on-line.  Some  students  were  given  a particular  time  and  location  for  the  chat,  while  other 
students  who  had  subscriptions  to  America  Online  were  left  to  arrange  a time  with  the  expert  at  their  convenience. 

Results  with  Synchronous  Chat 

The  pairs  collaborated  for  times  ranging  from  30  minutes  to  one-and-a-half  hours,  often  immediately  before  their 
scheduled  chat  time.  In  all  observed  cases,  one  student  from  each  pair  took  control  of  the  keyboard  for  the  30-minute,  on- 
line dialog.  With  other  issues,  a variety  of  approaches  to  the  chat  technology  were  observed.  A few  students  typed  many 
lines  of  background  information  before  any  interchanges  with  the  expert  took  place.  Others  would  address  a point  at  a 
time,  or  would  simply  respond  to  questions  placed  by  the  expert. 

Sentences,  or  single  ideas,  were  quite  often  broken  into  several  entries,  obviously  in  response  to  the  limitation  of  the 
interface.  And,  due  to  the  time  delay  in  posting  these  two-line  responses,  the  order  of  display  would  not  reflect  the  order 
of  conversation.  Figure  2 is  an  example  dialog  with  this  characteristic. 


Expert:  I have  produced  Hazardous  Communications  videos  and  videodiscs  for  several  companies 

Expert:  but  usually  they  are  only  concerned  with  labelling  and  safety  gear.  What  are  the  objectives 
Pair3:  Well  then,  I would  not  have  to  have  spent  so  much  of  my  precious  time  learning  this  editing  equipment 

Expert:  of  this  video. 

Pair3:  and  producing  this  video. 

Pair3:  Employees  will  be  able  to  have  acces  and  know  how  to  read  MSDS  sheets... 


Figure  2.  Example  synchronous  dialog. 

This  characteristic  appeared  to  confuse  both  the  novices  and  the  expert  when  significant  thoughts  were  left  apparently 
incomplete  until  many  moments  later.  The  trailing  notes  caused  students  to  re-read  earlier  entries  in  order  to  follow  the 
stream  of  the  dialog. 

Without  exception,  students  reported  a high  interest  in  the  novelty  of  the  chat  technology.  They  also  were  so  interested 
in  physically  meeting  the  expert,  that  the  class  took  a road  trip  of  1.5  hours  later  in  the  semester  to  meet  her. 

Procedure  with  World-Wide  Web 

We  are  just  beginning  to  use  the  World-Wide  Web  as  a medium  of  social  dialogue.  While  course  Materials  for  these 
classes  have  been  provided  on  the  web  for  over  a year,  the  materials  are  generally  information  only  and  do  not  lead  to 
dialog.  Social  discourse  through  the  Web  is  being  implemented  in  small  ways;  we  have  plans  to  implement  more  social 
discourse  in  the  future. 
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Presently,  some  questions  related  to  the  personal  experiences,  expectations,  and  contexts  of  the  course  are  being  asked  of 
students.  Students  respond  through  a web  page  with  fields  or  "forms".  The  responses  get  posted  » the  web  site  for  the 
course  for  all  other  students  to  browse.  Figure  3 is  an  example  of  one  the  questions  posed  through  the  web. 


Personal  and  job-related  context  related  to  this  course. 

Many  individuals  who  take  this  course  have  a computer  and  other  resources  already  available  to  them.  In  addition,  some 
individuals  have  an  immediate  need  for  computer-based  training  products  at  their  place  of  employment.  In  order  to 
improve  our  understanding  of  your  context,  please  respond  to  the  following  questions. 

• I have  personal  access  to  a computer  for  use  with  this  class. 

• I have  Internet  access  from  my  personal  computer. 

• I have  or  will  have  personal  access  to  the  ToolBook  authoring  system. 

•I  have  a project  already  selected  for  development. 

(Project  description  and  context): 

•For  this  course  I'd  like  to  convert,  some  portion  of  my  Internet 
course  to  a multimedia  format,  and  have  it  available  on  the  Web... 


Figure  3.  Example  Course  Questions  Posted  to  the  Web 

We  have  proposed  additional  software  to  take  advantage  of  the  multimedia  and  global  networking  capabilities  of  the  web. 
The  proposed  software  will  provide  instruction  and  support  in  the  domain  of  performance  technology.  Users  will 
investigate  industry-related  performance  problems  (e.g.  production  errors,  complaints,  low  production)  by  conducting 
"interviews"  and  recommending  solutions  (e.g.,  provide  training,  increase  rewards,  remove  obstacles.) 


Students  enrolled  in  courses  at  Penn  State  Harrisburg  will  be  solicited  for  participation  as  well  as  students  in  similar 
programs  at  other  institutions.  Recognized  experts  and  trainers  in  the  field  will  be  especially  encouraged  to  participate. 

Conclusions 

It  was  the  intention  of  the  study  to  implement  Internet  technologies  in  ways  that  support  and  promote  cognitive 
apprenticeships.  In  the  opinion  of  the  researchers,  this  was  not  fully  realized.  One  central  component  of  the  cognitive 
apprenticeship  model  is  multiple  practice  within  the  domain.  The  strategies  implemented  in  this  study  failed  to 
demonstrate  a form  of  multiple  practice  that  we  expected  to  see  in  social  discourse.  We  are  calling  this  feature  dialog 
momentum. 

Dialog  momentum  is  demonstrated  by  the  continuance  of  a discussion  upon  an  idea  through  several  rounds  of  responses. 
It  is  best  represented  in  electronic  communications  through  threads  of  discussions  with  multiple  listserv  postings  and 
personal  correspondence.  This  feature  was  not  well  demonstrated  in  this  study. 

Both  the  listserv  activities  and  the  synchronous  chat  activities  did  not  demonstrate  dialog  momentum  to  our  expectations. 
Listserv  discussions  did  not,  on  the  whole,  demonstrate  multiple  postings  concerning  single  topics.  Instead,  discussions 
revolved  around  only  a few  postings.  Within  the  chat  feature,  dialog  certainly  did  consist  of  multiple  responses. 
However,  the  disjuncted  nature  of  the  dialog  meant  that  one  had  to  constantly  move  backward,  not  forward  in  the 
discussion. 
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One  of  the  major  reasons  that  dialog  momentum  did  not  occur  in  the  listserv,  in  the  opinion  of  the  researchers,  is  the 
result  of  the  context  of  the  study.  Most  participants  had  access  to  electronic  mail  only  once  per  week  as  they  attended 
their  weekly  class.  Dialog  momentum  requires  a faster  turnaround  of  responses.  Additionally,  the  synchronous  chat 
session  would  have  been  greatly  improved  by  faster  response  times.  Slow  communication  response  times  was  the  major 
obstacle  to  moving  the  dialog  forward. 

In  summary,  present  Internet  communications  such  as  listservs  and  synchronous  chat  sessions  can  be  used  to  support 
:nstructional  purposes.  The  use  of  on-line  experts  can  aid  the  process  by  providing  additional  resources  and  points  of 
view.  True  cognitive  apprenticeships  which  require  repetitive  practice  through  the  notion  of  dialog  momentum  are 
difficult  to  accomplish.  More  attention  toward  creating  dialog  momentum  may  deliver  better  results. 
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Robert  Gagne’s  theories  and  research  in  instruction  and  learning  have  been  discussed  in  other  chapters,  where 
their  relationships  to  each  other  are  explored  in  depth.  Gagne's  theories  and  research  have  had  significant  impact  on 
practitioners  in  general  and  of  instructional  designers  in  specific,  and  this  will  be  the  focus  of  this  paper.  Given  the 
length  of  his  professional  career,  and  the  esteem  with  which  his  numerous  publications  are  held,  it  is  axiomatic  that  he 
has  had  an  impact.  Further  examination  reveals  that  he  also  has  influenced  teaching  and  curriculum  development  through 
his  research  and  theory.  He  also  used  standard  practices  as  a stimulus  for  the  development  of  theory.  Throughout  his 
career,  Gagne  was  always  cognizant  of  the  gap  between  theory  and  practice,  and  addressed  this  gap  by  directing  his 
investigations  toward  practical  problems. 

I personally  have  felt  the  impact  of  Robert  Gagne  during  my  military  training  in  the  late  1950s  and  early  1960s. 
These  were  my  earliest  experiences  as  an  adult  learner  and  a teacher  of  adults.  Reviewing  the  Gagne  literature  for  this 
paper  confirmed  a suspicion  I’ve  had  since  these  military  experiences.  There  seems  not  only  to  be  a possibility,  but  also 
a high  probability,  that  my  training  was  influenced  in  no  small  measure  by  Gagne  and  his  associates  in  the  military.  I 
experienced  first  hand  the  effectiveness  of  military  training  based  upon  Gagne’s  principles  as  both  a trainer  and  trainee. 
However,  my  interest  in  Gagne's  influence  on  practice  is  more  than  an  outgrowth  of  my  military  experience.  That,  in 
conjunction  with  25  years  as  a practicing  instructional  designer  in  public  schools,  colleges  and  industry  has  created  a 
somewhat  personal  relationship  to  Gagne  and  his  contributions. 

This  paper  will  explore  Gagne’s  influence  on  practice  by  first  examining  the  relationship  between  theory  and 
practice  especially  in  relation  to  instructional  design,  and  then  discussing  curriculum  development  and  transfer  of 
learning. 

The  Relationship  between  Theory  and  Practice 

Gagne  typically  examined  the  interaction  and  dependencies  between  theory  and  practice.  This  is  noteworthy 
given  the  attention  that  the  application  of  theory  to  practice  has  also  received  by  other  researchers.  The  work  of 
Huberman  (1990),  London  (1973),  Battersby  (1987),  Clark  (1988),  Schon  (1987),  and  Willis  (1993),  are  but  a few 
examples  of  researchers  who  have  joined  the  ranks  of  researchers  who  argue  that  good  theory  should  be  applied  to 
practice,  and  conversely  exemplary  practice  should  be  examined  as  a basis  for  new  theory  development.  Huberman  (1990) 
goes  further  in  linking  theory  to  practice  by  suggesting  that  researchers  should  start  their  research  by  first  contacting 
practitioners.  Furthermore,  he  notes  that  ”.  . . research  findings  can  flow  into  practitioner  settings  and  craft  knowledge 
can  move  into  research  settings  as  a natural  function  of  the  ongoing  relationships  between  both  parties  feeding  more  or 
less  automatically  into  their  customary  transactions”  (p.  387).  This  kind  of  relationship,  although  described  by 
Huberman  in  1990,  seems  to  reflect  many  situations  described  by  Gagne  in  his  early  work.  For  example,  Gagne  (1962), 
in  his  article  "Military  Training  and  Principles  of  Learning”  discusses  the  differences  between  those  learning  principles 
studied  in  laboratories  and  their  application  to  military  training,  and  recognizes  the  difficulties  of  applying  theory  to 
practice. 


I am  not  asking,  how  can  a scientific  approach  be  applied  to  the  study  of  training?  Not  am  I asking  how  can 
experimental  methodology  be  applied  to  the  study  of  training?  The  question  is,  rather,  how  can  what  you  know 
about  learning  as  an  event,  or  as  a process  be  put  to  use  in  designing  training  so  that  it  will  be  maximally 
effective?  (p.  84) 

Gagne,  as  early  as  the  late  1950s  and  early  1960s,  had  clearly  established  an  interest  in  and  desire  to  apply  theory 
to  practice.  He  was  especially  interested  in  examining  the  larger  issue  of  applying  theory  to  training,  teaching,  and 
learning  with  the  eventual  objective  of  making  it  more  effective  and  efficient. 

Gagne’s  early  observations  in  military  training,  research,  and  academic  laboratories  provided  ample  evidence  of 
the  inadequacy  of  existing  learning  theories  and  principles  as  vehicles  for  solving  pressing  training  problems,  and  the 
impact  of  his  reactions  to  these  observations  was  profound.  For  example,  his  theory  and  research  findings  were  applied 
to  the  development  of  training  on  trouble  shooting  aircraft  electrical  systems  and  electronics.  This  is  one  specific  area  in 
which  I experienced  training  bearing  the  earmark  of  Gagne’s  theories.  The  trouble  shooting  training  I participated  in  (and 
later  taught)  was  carefully  sequenced  hierarchically,  and  component  tasks  were  intended  as  mediators  that  directed  the 
instruction  and  learning  process  toward  the  ultimate  objective.  Military  trainees  were  evaluated  for  mastery  of 
prerequisite  skills  and  taught  or  retaught  these  skills  where  necessary.  All  of  these  strategies  reflect  the  influence  of 
Robert  Gagne. 


Gagne's  ever  present  concern  with  practice,  even  in  the  midst  of  theory  development,  continues  to  benefit 
education  and  training.  These  benefits  will  be  explored  here,  especially  in  terms  of  curriculum  development,  instructional 
design  practice  and  transfer  of  training. 

Impact  of  Gagne's  Theories  on  Curriculum  Development  Practice 

An  examination  of  curriculum  and  curriculum  development  logically  begins  with  a concept  definition.  Gagne 
(1966)  defines  curriculum  as 

...  a sequence  of  content  units  arranged  in  such  a way  that  the  learning  of  each  unit  may  be  accomplished  as  a 
single  act,  provided  the  capabilities  described  by  specified  prior  units  (in  the  sequence)  have  already  been 
mastered  by  the  learner,  (p.  22) 

This  orientation  is  a logical  extension  of  his  cumulative  learning  theory  and  his  notion  of  the  learning  hierarchy. 

Contrasting  definitions  illustrate  the  diversity  of  thinking  in  this  area.  For  example.  Bloom  (1976)  views 
curriculum  as  occurring  in  two  forms-- visible  and  invisible.  The  former  being  the  school  subjects  one  is  taught,  and  the 
latter  being  those  lessons  which  teach  one  his  or  her  place  in  the  world.  Gagne's  concept  is  closer  to  the  first  view. 

Further  contrasting  definitions  are  offered  by  Bruner  (1966)2 3,  Eisner  (1985)\  and  Klein  (American  Society  for 
Curriculum  Development,  1993)4.  Bruner  and  Klein  provide  views  that  are  more  traditional  and  closer  to  that  of  Gagne. 

Eisner,  on  the  other  hand,  also  recognizes  the  existence  of  both  formal  and  informal  curricula,  similar  to  Bloom.  While 
not  all  theorists  agree  on  the  definition  of  curriculum,  Gagne's  position  has  been  used  as  the  basis  for  a number  of 
important  efforts  in  schools  and  training. 

School  program  design.  The  most  pervasive  example  of  an  application  of  Gagne's  theories  and  research  to  a 
large  scale  curriculum  project  is  Science:  A Process  Approach  (SAP A),  which  is  part  of  the  American  Association  for  the 
Advancement  of  Science  (AAAS)  Commission  on  Science  Education.  These  science  curriculum  materials  were 
influential  in  schools  and  colleges  during  the  1960s  and  early  1970s  and  represent  a significantly  large  scale  curriculum 
effort  utilizing  Gagne’s  theories  and  research  in  the  areas  of  problem  solving  and  scientific  inquiry.  Gagne’s  view  of  a 
process  approach  to  science  is  scientific  inquiry  and  is  based  on  students  having  a large  knowledge  base  which  they 
subsequently  utilize  to  make  and  then  test  inductive  inferences.  The  underlying  foundation  for  the  process  approach  is 
hierarchical,  and  presumes  that  learners  have  the  prerequisite  process  skills  as  background  Gagne  (1965)  maintained  that 

2Bruner's  (1966)  definition  is  "A  curriculum  should  involve  the  mastery  of  skills  that  in  turn  lead  to  the  mastery  of  still  more 
powerful  ones,  the  establishment  of  self-reward  sequences. 

...  a curriculum  should  be  prepared  jointly  by  the  subject  matter  expert,  the  teacher,  and  the  psychologist  with  due  regard  for 
the  inherent  structure  of  the  material,  its  sequencing,  the  psychological  pacing  of  reinforcement  and  the  building  and 
maintaining  of  predispositions  to  problem  solving  (pp.  35,  70). 

3Eisner  (1985)  defines  three  curriculums:  null,  implicit  and  explicit. 

Null:  A curriculum  in  which  all  of  those  things  not  taught  and  not  learned  in  schools-there  simply  are  no  opportunities  to 
learn  them. 

Implicit:  (This  is  similar  to  Bloom's  latent  curriculum).  A curriculum  in  which  ideas,  values,  attitudes,  and  processes  are 
not  explicitly  taught;  but  they  are  none-the-less,  learned.  They  are  learned  through  the  subtleties  of  teacher  values  and 
attitudes,  as  well  as  the  signals  sent  by  the  organization  as  a whole  (e.g.,  where  it  puts  its  resources,  and  what  it  values— 
sports,  academics,  fine  arts,  etc.) 

Explicit:  The  curriculum  to  which  students,  teachers  and  administrators  must  attend  to  most  in  schools.  It  is  what  parents 
and  society  expect  students  to  have  learned,  and  what  they  try  and  measure  as  predictors  of  success.  This  curriculum  offers 
tangible  evidence  of  its  existence  through  instructional  materials,  technology,  instructional  strategies,  guides,  etc.  The 
explicit  curriculum  is  often  perceived  as  that  cumulative  knowledge  of  human  kind  which  is  passed  on  through  the 
generations. 

4Klein's  (American  Society  for  Curriculum  Development,  1993)  definition  is  "Curriculum  is  those  activities,  processes  and 
structural  arrangements  as  intended  for,  employed  in,  or  experienced  in  the  school  and  classroom  for  the  purposes  of  fulfilling 
the  educative  function  (p.  2.16). 
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the  process  approach  is  a middle  ground  between  the  "content  approach"  and  the  "creative  approach"  and  "It  substitutes  the 
notion  of  having  children  learn  generalizable  process  skills  which  are  behavioral  specific,  but  which  carry  the  promise  of 
broad  transferability  across  many  subject  matters"  (p.  4).  It  can  also  be  said  that  SPAP  and  its  orientation  to  teaching 
elementary  science  and  scientific  inquiry,  although  first  published  in  the  sixties,  remained  immensely  influential  in 
science  texts  and  other  commercially  published  science  materials  well  into  the  1980s.  Andrew  Ahlgren  of  AAAS,  co- 
author of  Science  for  All  Americans,  provided  furtl  r testimony  to  Gagne's  influence  on  science  curriculum,  as  well  as 
his  indirect  influence  on  mathematics,  and  technology  curriculum  in  specific  (A.  Ahlgren,  October  3,  1994,  personal 
communication).  He  stated  that  SAPA  most  certainly  had  tremendous  influence  on  not  only  science,  but  also  technology 
curriculum. 

Not  all  see  Gagne's  influence  on  science  curriculum  as  positive.  Finiley  (1983),  for  example,  argues  that 
Gagne's  theories,  as  well  as  others  of  like  mind,  have  propelled  science  curriculum  in  the  wrong  direction  by  advocating  a 
commitment  to  inductive  empiricism.* * * * 5  He  maintains  that  a presentation  of  papers  by  Gagne  to  AAAS  . . has  had  a 
substantial  influence  on  curriculum,  instruction,  and  research  in  science  education  since  that  presentation"  (p.  47). 

Finiley  then  selects  Gagne,  in  view  of  all  others  writing  about  science  process,  as  the  most  influential  when  he  says: 
"Although  many  science  educators  have  written  about  science  processes,  the  view  established  by  Gagne  has  been  most 
influential"  (p.  48).  He  continues  his  argument  from  a philosophical  perspective  indicating  that  Gagne,  similar  to  his 
predecessors  like  Francis  Bacon,  Robert  Boyle,  Sir  Isaac  Newton  and  Hume,  embrace  the  positions  of  empiricism  and 
induction.  Finiley,  although  in  fundamental  disagreement  with  Gagne's  approach  to  teaching  science,  substantiates  the 


5Finiley,  when  discussing  Gagne's  theories  is  making  direct  reference  to  Gagne's  influence  on  science  curriculum  through 
AAAS  in  general  and  SPAP  in  specific.  The  influence,  as  mentioned  earlier,  is  centered  around  Gagne's  perspective  of  a 
process  approach  where  learners  are  taught  to  think  and  solve  problems  like  a scientist  would.  In  Gagne's  (1965)  scheme  this 
would  be  accomplished  b5Bruner's  (1966)  definition  is  "A  curriculum  should  involve  the  mastery  of  skills  that  in  turn  lead 
to  the  mastery  of  still  more  powerful  ones,  the  establishment  of  self-reward  sequences. 

...  a curriculum  should  be  prepared  jointly  by  the  subject  matter  expert,  the  teacher,  and  the  psychologist  with  due  regard  for 
the  inherent  structure  of  the  material,  its  sequencing,  the  psychological  pacing  of  reinforcement  and  the  building  and 
maintaining  of  predispositions  to  problem  solving  (pp.  35,  70). 

5Eisner  (1985)  defines  three  curriculums:  null,  implicit  and  explicit. 

Null:  A curriculum  in  which  all  of  those  things  not  taught  and  not  learned  in  schools—there  simply  are  no  opportunities  to 
learn  them. 

Implicit:  (This  is  similar  to  Bloom's  latent  curriculum).  A curriculum  in  which  ideas,  values,  attitudes,  and  processes  are 
not  explicitly  taught;  but  they  are  none-the-less,  learned.  They  are  learned  through  the  subtleties  of  teacher  values  and 
attitudes,  as  well  as  the  signals  sent  by  the  organization  as  a whole  (e.g.,  where  it  puts  its  resources,  and  what  it  values- 
sports,  academics,  fine  arts,  etc.) 

Explicit:  The  curriculum  to  which  students,  teachers  and  administrators  must  attend  to  most  in  schools.  It  is  what  parents 
and  society  expect  students  to  have  learned,  and  what  they  try  and  measure  as  predictors  of  success.  This  curriculum  offers 
tangible  evidence  of  its  existence  through  instructional  materials,  technology,  instructional  strategies,  guides,  etc.  The 
explicit  curriculum  is  often  perceived  as  that  cumulative  knowledge  of  human  kind  which  is  passed  on  through  the 
generations. 

5Klein's  (American  Society  for  Curriculum  Development,  1993)  definition  is  "Curriculum  is  those  activities,  processes  and 
structural  arrangements  as  intended  for,  employed  in,  or  experienced  in  the  school  and  classroom  for  the  purposes  of  fulfilling 
the  educative  function  (p.  2.16). 

5Finiley,  when  discussing  Gagne's  theories  is  making  direct  reference  to  Gagne's  influence,  on  science  curriculum  through 
AAAS  in  general  and  SPAP  in  specific.  The  influence,  as  mentioned  earlier,  is  centered  around  Gagne's  perspective  of  a 
process  approach  where  learners  are  taught  to  think  and  solve  problems  like  a scientist  would.  In  Gagne's  (1965)  scheme  this 
would  be  y learning  prerequisite  skills  which  transfer  to  more  complex  skills,  and  eventually  lead  the  learner  to  a level 
where  they  are  able  to  carry  out  scientific  thinking  which  is  disciplined  and  systematic  and  connected  to  . . the  process  of 
science"  (p.  4). 
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overreaching  influence  Gagne  has  had  on  the  development  of  science  through  SAPA  during  the  late  1960s  and  into  the 
1980s. 

Hackett  (1971)  provides  another  example  of  the  use  of  Gagne's  theories  on  a large  scale  curriculum  project  in  a 
public  school  setting.  Although  her  work  was  primarily  directed  toward  reading  and  communication  skills  curricula,  she 
provides  ample  evidence  of  the  application  of  Gagne's  theories  to  social  studies  and  mathematics  as  well.  Hackett's 
experiments  and  curriculum  projects  focused  on  a performance-based  approach  which  has  many  similarities  to  the 
outcome  based  education  movement  of  the  late  1980s  and  early  1990s. 

There  are  also  many  examples  of  smaller  scale  curriculum  efforts  that  apply  Gagne's  theory  to  curriculum 
development  projects.  Two  examples  are  Gilbert's  (1992)  use  of  Gagne's  hierarchies  in  his  curriculum  on  questioning 
and  taxonomies,  and  Lines’s  (1988)  work  with  advanced  economics.  These  programs  provide  evidence  of  more  recent 
applications  of  Gagne’s  theories  to  curriculum.  One  can  also  examine  as  evidence  Margaret  E.  Bell's  (1982)  article  in 
which  she  makes  a persuasive  case  for  the  application  of  Gagne's  theories  to  designing  programs.  She  argues  that 
curriculum  design  and  development  has  not  been  as  systematic  as  the  efforts  of  designing  instruction.  Bell  recommends 
that  Gagne’s  five  capabilities  can  be  applied  to  course  instruction  as  well  as  program  or  curriculum  development.  John 
Flynn’s  (1992)  also  adapts  Gagne’s  Events  of  Instruction  to  the  very  high  profile  and  contemporary  research  area  of 
cooperative  learning. 

School  lesson  design.  When  relating  Gagne’s  theories  to  curriculum  efforts  that  are  directed  toward  individual 
lessons  many  of  the  examples  utilize  computer  technology.  Lesgold's  (1987)  effort  wherein  goal  knowledge  was 
examined  as  to  its  significance  to  ”. . . intelligent  machine  . . . [and]  human  activity  ...  is  an  example  of  adapting 
Gagne’s  theories  to  curriculum  and  prerequisite  skills  in  a novel  way.  Also  in  this  category  is  the  Smaldino  and 
Thompson  (1990)  research  relating  the  Events  of  Instruction  to  science  education  and  computer  technology.  These 
authors  propose  designing  science  lessons  focusing  on  the  "Nine  Events  of  Instruction  (p.  17).  Jonassen  (1988)  has 
utilized  many  of  Gagne's  writings,  theories  and  principles  in  the  design  of  microcomputer  courseware.  He  especially 
utilizes  Gagne’s  Events  of  Instruction  and  his  work  in  the  area  of  hierarchies  and  prerequisite  skills.  Jonassen  (1988)  also 
utilizes  Gagne's  work  with  respect  to  learning  outcomes  in  designing  individual  lessons  to  be  delivered  by  computer 
courseware. 

Training  curriculum  design.  Gagne's  theories  also  have  been  used  extensively  in  training  curriculum  design  in 
the  private  sector  of  business,  or  the  non-school  sector  of  governmental  agencies.  It  is  most  appropriate  to  start  with  the 
military  and  defense  related  environments  where  the  evidence  of  Gagne's  influence  abounds.  While  many  of  Gagne  s early 
writings  are  generously  sprinkled  with  references  to  military  applications  and  research  results  conducted  in  military 
settings,  there  are  also  many  current  applications  made  in  private  sector  training.  Stepich  (1991)  and  Garavaglia  (1993) 
provide' two  examples.  Stephich  (1991)  examines  the  idea  of  utilizing  training  to  move  learners  from  novice  to  expert 
status,  and  proposes  a way  to  apply  Gagne's  "conditions  of  learning'  to  training  design.  Garavaglia  (1993)  suggests  that 
designers  take  another  look  at  the  design  phase  of  Instructional  Systems  Development  (ISD).  Garavaglia  contends  that: 
"For  each  event  of  instruction  you  should  determine  the  method  for  which  it  can  be  achieved  and  the  media  necessary  to 
achieve  it"  (p.  28).  He  continues  by  expanding  on  how  the  Events  of  Instruction  can  be  used  in  conjunction  with  Kellers 
ARCS  model  in  what  Garavaglia's  calls  a technical  training  submethodology.  Both  of  these  articles  utilize  Gagne's 
theories  to  develop  techniques,  methods  or  practices  and  each  imply  that  the  practice  based  upon  his  theories  has 
implications  for  a larger  curriculum  effort  throughout  a training  program  in  the  private  sector. 


Impact  of  Gagne's  Instructional  Design  Theories  on  Instructional  Design  Practice 


The  profound  influence  of  Gagne's  theories  on  instructional  design  practice  is  most  easily  understood  when 
positioning  them  into  the  context  of  his  early  theories  of  instruction  or  learning.6  It  is  important  to  realize  that  he  was 

fiThe  influence  of  Gagne's  theories  on  instructional  design  practice  reveals  two  paths  which  are  not  only  interrelated,  but 
nearly  inseparable.  The  first  path  is  an  analysis  of  textbooks  and  handbooks.  The  instructional  design  textbooks  and 
handbooks  serve  many  purposes,  one  of  which  is  to  provide  a communication  link  between  theory,  research  and  the  practice 
of  instructional  design  in  education,  educational  psychology  and  training.  The  argument  here  is  that  there  are  two  important 
goals  for  texts  and  handbooks.  First,  to  communicate  to  the  readers'  the  theories  and  models  found  within  them;  and  second, 
to  promote  these  theories  in  the  respective  practices  of  education  and  training  of  their  readers. 

The  second  path  is  an  examination  of  the  practical  research  reports,  journal  articles  and  curriculum  projects  which 
report  the  findings  of  research,  theory  development  and  application  to  instructional  design  practice.  These  resources  are  more 
likely  to  be,  in  relation  to  this  paper,  reports  on  the  application  of  Gagne's  instructional  design  theories  to  specific  content 
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one  of  the  theorists  instrumental  in  bridging  the  gap  between  the  behaviorists  of  the  1950s  and  1960s  and  the 
cognitivists  of  the  1970s  and  1980s.  Case  and  Berither  (1984)  maintain  that  when  Gagne  ".  . . shifted  the  focus  of 
attention  from  the  how  to  the  what  of  behavior  change;  that  is,  he  shifted  the  focus  from  reinforcement  to  the  nature  of 
the  behaviours  themselves"  (p.  144). 

Case  and  Bereither  (1984),  suggest  that  Gagne  not  only  moved  away  from  reinforcement,  but  he  also  recognized 
learning  as  a more  complex  process  than  previously  thought,  and  they  elaborate  on  Gagne's  recognition  that  learning  was 
not  confined  to  . . the  learning  of  physical  behaviors  and  simple  stimulus-response  connections  but  also  the  learning 
of  concepts,  rules,  principles,  intellectual  skills  and  cognitive  strategies"  (p.  144).  Using  Gagne's  earlier  work  as 
background,  they  suggest  that  the  third  and  most  important  part  of  his  work,  which  catapulted  him  beyond  the 
behaviorists  of  that  time,  was  his  concept  of  sequencing  intellectual  skills  and  allowing  the  instruction  to  move 
systematically  toward  higher-order  skills  all  while  building  on  prerequisite  skills. 

Gagne  had  a part  in  the  paradigm  shift  from  behavioral  to  cognitive  psychology  in  the  early  1960s,  and  this 
brought  about  a predictable  change  in  both  instructional  design  literature  and  practice.  The  literature  of  the  field,  viewed 
as  a communication  link  or  mediator  between  theory  and  practice,  is  certainly  a measure  of  just  how  pronounced  his 
influence  has  been.* * * * * * 7 

Richey  (1986)  maintains  that  Gagne  has  had  tremendous  influence  on  instructional  design  practice  through  his 
theories,  models  and  procedures  for  developing  instruction  Instructional  designers  have  embraced  Gagne's  theories  for 
many  reasons;  however,  one  of  the  most  compelling  reasons  lies  in  his  work  with  learning  outcomes.  Gagne  (1988) 
directs  the  instructional  designer  to  utilize  the  following  learner  outcomes  when  analyzing  content:  intellectual  skills,, 
verbal  information,  cognitive  strategies,  motor-skills  and  attitudes.  Subsequent  to  determining  the  desired  learning 
outcome,  the  instructional  designer  is  advised  to  complete  the  content  analysis  based  on  the  expectations  for  the  learner. 
The  documentation  of  the  design  process  where  the  designer  selects  the  appropriate  learner  outcomes,  completes  the 
content  analysis  and  develops  the  appropriate  flow  diagrams  and  procedures  becomes  the  core  of  the  instructional  design 
document  used  to  guide  the  instructional  design  project  to  completion. 

No  examination  of  Gagne's  influence  on  practice  would  be  complete  without  examining  the  influence  of  his 
theories  on  teacher  education  and  ultimately  on  teachers,  professors  and  the  entire  education  enterprise.  Furthermore,  this 
examination  compels  the  researcher  to  delve  further  into  the  definition  and  concepts  of  influence  and  change.  Short  term 


areas  or  disciplines  within  the  educational  or  training  arena.  Since  the  author  devoted  considerable  sp  f>e  and  cited  several 

sources  from  this  path  when  examining  Gagne's  influence  on  curriculum,  no  further  effort  will  be  made  to  elaborate  on  these 

items.  It  is  clear  however,  that  Gagne's  impact  on  instructional  design  practice  is  evident  in  the  enormous  number  of  journal 

articles,  research  reports  and  curriculum  projects  which  refer  to  his  work.  The  reader  is  well  advised  to  pursue  these  resources 

or  to  refer  to  the  curriculum  section  of  this  paper  for  further  information.  No  specific  effort  has  been  made  to  identify  when 

resources  from  one  or  another  path  are  being  highlighted  other  than  to  identify  items  as  books,  reports  or  journal  articles. 

7Textbooks  and  handbooks  are  a primary  communication  link  between  theory  and  practice,  and  as  such  they  are  an  essential 
resource  for  measuring  Gagne's  influence.  Among  his  texts  are  the  following:  the  four  editions  of  Gagne’s  The  Conditions 
of  Learning  (1965),  the  two  editions  of  Gagne  and  Briggs  (1974  & 1979),  Principles  of  Instructional  Design  and  the  third 
edition  of  the  same  title  by  Gagne,  Briggs  and  Wager  (1988).  These  books  alone  would  indicate  a monumental  impact  on 
instructional  design  practice  since  they  are  cited  throughout  the  instructional  design  literature  that  parallels  them,  and  almost 
all  the  instructional  design  literature  that  follows.  They  are  also  texts  from  which  many  instructional  design  practitioners  in 
the  1970s  and  1980s  learned  the  theory  and  practice  of  instructional  design. 

There  are  several  series  of  texts  which  further  explain  and  apply  Gagne’s  theories  for  practitioners.  An  example  of  a 
multiple  series  of  texts  is  Dick  and  Carey’s  (1978,  1985,  1990),  The  Systematic  Design  of  Instruction,  with  a fourth  edition 
in  press.  There  are  very  few  practitioners  anywhere  haven’t  taken  a course  where  this  text  was  used,  applied  it  to  their 
practice,  taught  from  it,  or  at  least  read  it  in  part.  Although  the  Dick  and  Carey  editions  can  be  characterized  many  ways,  they 
are  theoretically  "vintage  systems  theory"  with  the  strong  influence  of  Gagne  in  their  application  of  instructional  design 
theory  to  practice.  They  also  model  Gagne’s  desire  to  be  practical  by  presenting  their  system  for  designing  instruction  as  one 
that  accommodates  either  a "knowledge"  or  "product"  approach.  They  add  that  they  favor  the  product  approach  since  it  requires 
students  to  actually  develop  instruction  as  opposed  to  learning  about  instructional  design  as  a theoretical  concept.  There  are 
many  other  series  and  single  texts  which  have  been  influenced  by  Gagne's  theories  and  research,  and  one  worthwhile 
consulting  to  this  end. 


change  in  attitudes  brought  about  in  pre-service  educators  being  exposed  to  Gagne’s  theories  in  methods  and 
media/technology  courses  may  be  assessed  traditionally  in  course  evaluations  and  tests;  however,  expecting  them  to 
incorporate  these  theories  and  concepts  into  their  teaching  practice  is  a different  matter.  This  is  especially  true  when 
considered  from  the  perspective  of  initiating  permanent  change  on  professionals  who  ultimately  spend  their  careers  in  an 
organizational  culture  which  has  many  years  of  history,  precedence,  and  accepted  methodology,  which  often  reinforces  the 
attitudes  of  experienced  teachers,  and  shapes  the  attitudes  of  new  teachers.  Martin  and  Clemente  (1990)  argue  that 
instructional  systems  design  (ISD)  has  had  minimum  impact  on  schools  because  professionals  as  well  as  professors  of 
ISD  haven’t  considered  carefully  enough  their  clients  (teachers)  and  the  culture  of  schools.  They  further  argue  that  until 
we  (ISD  professionals)  understand  that  the  accepting  of  the  ISD  approach  would  be  considered  an  innovation  in  schools; 
therefore,  subject  to  all  the  usual  barriers  to  accepting  change  in  the  culture  or  social  setting  we  will  be  unsuccessful  in 
promoting  ISD  in  schools. 

Are  we  to  conclude  then  that  Gagne's  theories  have  had  little  or  no  influence  on  teachers,  professors  and 
subsequently  education:?  Not  necessarily!  While  it  is  true  that  there  has  not  been  a preponderance  of  teachers  or 
educational  systems  adopting  ISD  in  its  entirety,  there  is  evidence  that  teachers  adapt  ISD  theories  to  their  practice.  In  so 
far  as  Gagne's  influence  is  concerned  these  adaptations  are  frequently  made  to  ISD  theories  that  often  have  his 
instructional  theories  embedded  within  them.  A persuasive  argument  can  be  made  that  in  implicit  and  often  small  ways, 
over  an  extended  time  period,  teachers  and  professors  exposed  to  ISD  theories,  models  and  procedures  adapt  them  to  their 
practice  rather  than  adopt  them.  These  adaptations  bear  resemblance  to  both  Gagne  and  general  systems  theory.  Planning 
proceeds  from  problem  identification  writing  objectives  and  flow  charting  to  sequencing  content  and  concepts  from 
simple  to  complex  forming  the  basis  for  using  lower  level  intellectual  skills  as  a foundation  for  higher  order  skills, 
culminating  with  assessment.  Given  that  this  argument  is  lodged  on  the  premise  that  the  texts  used  in  educational  media 
and  instructional  technology  courses  for  preservice  teachers  have  been  and  continue  to  be  strongly  influenced  by  general 
ISD  theories  which  have  Gagne’s  theories  embedded  in  them,  it  is  recommended  that  the  reader  examine  these  texts  and 
their  potential  in  relationship  to  Gagne.8 

Finally,  teachers  at  the  K-12  level,  as  well  as  those  at  the  college  and  university  level,  have  tremendous 
responsibility  for  instructional  design  albeit  much  less  formal  than  professional  designers  might  be  accustomed  to  or 
prescribe.  Therefore,  the  exposure  to  instructional  design  this  group  receives  while  participating  in  education  courses 
utilizing  the  theories  of  Gagne  and  other  theorists  has  the  potential  to  significantly  influence  the  course  and  unit  design 
these  practitioners  engage  in  throughout  their  careers.  The  large  number  of  pre-service  teachers  in  the  United  States  who 
take  media  and  methods  or  instructional  technology  courses  provides  evidence  for  the  potential  impact  of  these  teachers 
adapting  Gagne's  theories  and  models  in  their  practice.  Although,  as  mentioned  earlier,  the  adapted  instructional  design 
practiced  by  teachers,  instructors  and  professors  is,  on  the  surface,  often  less  systematic  and  formalized  than  the 
instructional  design  models  recommended  in  Gagne’s  books,  nonetheless  it  does  qualify  as  designing  instruction. 
Instructional  design  theorists  and  researchers  would  be  well  advised  to  consider  preservice  teachers  and  their  professors  as 
one  of  the  many  types  of  instructional  designers.  In  so  doing  the  texts  written  for  them  might  acknowledge 
organizational  differences  within  which  these  professionals  work  and  subsequently  influences  how  they  design 


“The  flagship  texts  in  terms  of  practical  application  to  preservice  education  are  the  Heinich,  Molenda,  and  Russell  editions  of 
Instructional  Media  and  the  New  Technologies  of  Instruction  (1982,  1985,  1989,  1993).  The  third  edition  of  the  Heinich  et 
al.  is  the  first  to  mention  Gagne’s  theory  explicitly  as  widely  accepted  and  recommended  for  the  design  of  instruction; 
however,  there  is  also  ample  evidence  of  implicit  use  of  his  work  in  the  first  and  second  editions.  This,  and  other  books  in 
this  genre,  target  an  audience  of  preservice  teachers  primarily  at  the  undergraduate  level;  however,  they  are  also  used  at  the 
graduate  level  as  introductory  texts  in  teacher  education  and  instructional  media  programs.  Their  goals  focus  on  providing 
their  audiences  with  information,  techniques,  utilization,  ideas  and  examples  related  to  instructional  media  and  the  use  ot 
technology  in  education.  These  texts  have  had  influence  not  only  by  mentioning  Gagne’s  theories  (although  that  is 
significant),  but  also  by  the  fact  that  the  professors,  instructors  and  trainers  of  these  courses  use  these  texts.  As  professionals 
they  have  had  wide  exposure  to  Gagne’s  theories  through  their  education  as  well  as  teaching  multiple  courses  utilizing  his 
work.  These  books  often  apply  and  model  Gagne’s  theories  and  writings  at  least  in  implicit  ways,  and  sometimes  explicitly. 
This  is  clearly  the  case  in  the  third  and  fourth  editions  of  Heinich  et  al.  (1989,  pp.  42  & 53)  wherein  they  cite  Gagne  and 
Briggs  (1988)  and  Gagne  (1985).  Furthermore,  the  instructional  design  model  recommended  in  these  texts,  and  especially  in 
the  third  and  fourth  editions,  has  many  elements  of  Gagne's  theories  and  is  similar  to  other  Gagne  instructional  design 
models. 


instruction.  Furthermore,  these  texts  might  then  incorporate  only  the  most  practical  and  useful  aspects  of  design  theory 
from  researchers  such  as  Gagne. 

Summarizing,  the  influence  of  Gagne's  theories  on  instructional  design  practice  spans  a gap  from  a reliance  on 
behaviorism  as  a foundational  theory  to  the  eventual  adoption  of  cognitivism  as  an  underlying  theory.  Further,  Gagne's 
overwhelming  influence  on  the  literature  read  by  practitioners  and  the  researchers  who  teach  them  has  had  significant 
impact  on  practice.  Finally,  the  indirect  or  implicit  influence  Gagne  has  had  on  the  informal  instructional  design 
practiced  by  teachers  and  many  professors  through  texts  for  preservice  education  is  greater  than  many  writers  realize. 

Gagne’s  Influence  on  the  Transfer  of  Learning 

When  reading  Gagne's  work,  and  especially  the  four  editions  of  The  Conditions  of  Learning,  one  is  impressed 
with  his  attention  to  detail  related  to  the  many  dimensions  of  learning  and  transfer.  Gagne  discusses  often  that  learning 
should  be  generalized  to  new  and  varied  content  and  applied  to  situations  in  the  learners  life.  Syllogistically,  the 
argument  could  be  made  that  through  the  four  editions  of  The  Conditions  of  Learning,  and  through  his  work  with  the 
Events  of  Instruction,  Gagne  always  had  been,  and  continues  to  be,  dedicated  to  both  near  and  far  transfer. 

Gagne  (1989)  was  experimenting  with  the  transfer  of  training  as  early  as  the  late  1940s.  This  early  research 
examines  positive  and  negative  transfer  and  discusses  transfer  in  the  context  of  giving:  . . different  amounts  of 

training  to  separate  groups  of  subjects  on  an  initial  task  which  was  a subordinate  part  of  a total  skill  involving  four 
differential  manual  reactions"  (p.  22).  This  research  was  done  with  training  subjects  on  complex  motor  tasks  using 
multiple  trials  and  observing  them  for  periods  of  little  or  no  improvement  (plateaus)  in  learning.  In  this  study  the 
control  group  performed  better  than  the  group  with  too  few  trials  (negative  transfer).  The  control  group  was  out- 
performed by  the  group  having  optimal  trials  (positive  transfer).  Positive  and  negative  transfer  are  defined  in  many  ways; 
the  following  eclectic  definitions  suit  this  discussion. 

Positive  transfer  occurs  when  learning  one  task  assists  in  the  performance  of  another;  or  when  a previously 

learned  task  enhances  the  ability  or  performance  in  another  task  or  control  group. 

Negative  transfer  occurs  when  the  learning  of  one  task  impairs  the  learning  of  another;  or  previously  learned  task 

is  an  impediment  to  performance;  or  experimental  group  is  outperformed  by  the  control  group. 

It  is  predictable  that  Gagne's  view  of  transfer  would  parallel  the  previous  definitions  at  that  time  in  his  career 
given  the  influence  of  early  behaviorists  like  Pavlov,  Guthrie,  Tolman,  Skinner  and  Thorndike  on  his  work.  Further,  his 
work  involving  positive  and  negative  transfer  became  the  basis  for  his  later  concept  of  transfer  which  has  been  so 
thoroughly  embedded  in  the  practice  of  instructional  design. 

The  discussion  that  follows  is  centered  on  Gagne's  evolving  concept  of  transfer  over  a 20  year  span.  During  that 
time  frame  his  use  of  the  term  was  modified  from  one  which  differentiated  between  positive  and  negative  transfer  to  the 
more  contemporary  lateral  and  vertical  concepts  found  throughout  the  literature  and  utilized  by  practitioners  today. 

Gagne  (1962)  in  later  references  to  transfer  builds  on  the  concepts  of  positive  and  negative  transfer.  When 
discussing  transfer  in  the  context  of  ". . . transfer  of  training  from  component  learning  sets  to  a new  activity  which 
incorporates  these  previously  acquired  capabilities"  (p.  364)  he  seems  to  be  directing  his  focus  more  toward 
generalization,  which  becomes  the  focus  of  his  later  conceptualization  and  subsequent  definition  of  transfer.  Gagne 
(1965),  when  discussing  external  events  and  the  conditions  of  learning  indicates  the  need  for  what  has  been  learned  to  be 
". . . generalizable,  and  transferable  . . ."  (p.  206)  to  new  and  different  situations  where  it  might  be  applied. 

Gagne  (1970)  says  that  capabilities  learned  in  school  should  provide  students  with  the  background  and  skills  to 
accomplish  practical  things  in  their  lives  or  in  occupations  and  identifies  this  as  lateral  transfer.  Furthermore,  he  says 
students  should  be  able  to  learn  more  complex  things  as  a result  of  their  previous  learning.  This  learning  of  more 
advanced  or  complex  tasks  or  skill  based  on  subordinate  rules  or  concepts  is  called  vertical  transfer.  The  defining  of 
lateral  and  vertical  transfer  within  the  framework  of  the  conditions  for  learning  helped  establish  the  foundation  for 
applying  the  concept  of  transfer  to  contemporary  instructional  design  practice. 

Some  of  Gagne's  lesser  known  theories  of  learning  and  instruction  are  both  an  indication  of  his  continued  search 
for  unique  ways  of  solving  learning  and  instructional  problems,  and  his  willingness  to  examine  the  contrasting  work  of 
other  researchers.  Gagne  (1968)  offers  the  cumulative  learning  theory  to  those  practitioners  having  some  difficulty  with 
total  acceptance  of  his  hierarchical  or  taxonomical  theories.  Although  hierarchical  in  the  pure  sense,  the  cumulative 
learning  theory  offers  a modified  approach  to  learning  and  transfer.  His  explanation  of  this  theory  begins  with  his 
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contrasting  two  models  of  intellectual  development,  one  by  Hall  and  Gessell  and  another  by  Piaget.  Gagne  also 
examines  two  kinds  of  capability  change,  both  of  which  are  observable  and  distinguishable  by  the  time  frame  required  for 
the  change  to  take  place.  Those  changes  in  behavior  capabilities  that  occur  in  hours,  days,  or  weeks  are  referred  to  as 
learning  and  memory;  those  behavior  capability  changes  requiring  months  or  even  years  are  called  development.  One  of 
the  many  questions  surrounding  the  difference  between  learning  and  development  is  that  each  view  transfer  of  learning 
from  a different  perspective. 

Additional  examination  of  Gagne’s  theories  related  to  human  development  reveals  his  Cumulative  Learning 
Model  and  its  relationship  to  transfer.  Gagne  maintains  that  using  this  model,  which  begins  with  S-R  connections,  and 
proceeds  through  concepts  and  simple  rules  and  culminates  with  complex  rules,  will  enhance  transfer.  In  other  words, 
learned  capabilities  at  any  stage  of  the  model  will  not  only  transfer  to  further  enhance  the  intended  learning  (i.e.,  the 
content  for  which  the  hierarchy  was  originally  designed),  but  may  also  serve  to  transfer  to  new  but  related  tasks.  Gagne 
(1988),  when  referring  to  "stages"  or  "levels"  in  reference  to  learning  new  material  related  to  previously  learned  material, 
states: 


Cumulative  learning  thus  assumes  a built  in  capacity  for  transfer.  Transfer  occurs  because  of  the  occurrence  of 

specific  identical  (or  highly  similar)  elements  within  developmental  sequences,  (p.  338) 

Gagne,  adds  that  "elements"  has  specific  meaning  in  this  discussion  of  transfer  since  it  refers  directly  to  . . 
rules,  concepts,  or  any  of  the  other  learned  capabilities  . . ."  (p.  338). 

However,  the  larger  question  here  is:  do  instructional  designers,  engaged  in  the  process  of  practicing  their  skills 
and  sel<  cting  learning  theories  automatically  consider  cumulative  learning  as  a theory?  The  author's  bias  leads  him  to  the 
conclusion  that  a large  percentage  of  practitioners  are  not  familiar  with  it  much  less  utilize  it  in  their  practice  to  enhance 
transfer.  Designers  familiar  with  theories  which  are  not  "mainstream"  whether  Gagne's  or  others  face  the  dilemma  of: 
going  back  to  what  they  know  best,  or  experimenting  with  fresh  approaches  at  a time  when  deadlines  are  shorter,  and 
there  is  increasing  pressure  to  decrease  the  time  of  the  design  cycle. 

When  considering  Gagne's  transfer  theories  and  their  relationship  to  intellectual  skills  and  higher  order 
capabilities,  it  is  evident  that  Gagne  accepts  the  proposition  that  intellectual  skills  and  higher  order  capabilities  may  be 
learned  for  a specific  intent  or  objective.  These  then  become  the  background  for  generalization  or  transfer  to  other  or  new 
learning.  The  generalizations  made  by  the  learners  may  be  a result  of  planned  instruction,  while  in  other  cases  they  may 
(based  on  need  or  curiosity)  take  the  initiative  to  learn  independently.  Since  learning  ascribed  to  this  theory  is 
cumulative,  it  often  becomes  more  complex  in  the  process  of  development;  therefore,  generalization  and  transfer  between 
and  among  those  things  already  learned  and/or  those  to  be  learned  is  enhanced.  Gagne  (1988)  when  commenting  on  the 
process  of  transfer  says:  "There  is  no  magic  key  to  this  structure-it  is  simply  developed  piece  by  piece.  The  magic  is  in 
learning  and  memory  and  transfer"  (p.  332). 

Summarizing  the  preceding  definitions  and  discussions  of  transfer  and  their  implications,  Gagne’s  perspective  is 
clear:  the  most  important  aspect  of  transfer  is  its  dependency  on  what  has  already  been  learned.  In  short,  there  is  nothing 
to  transfer  if  it  hasn't  already  been  learned.  The  second  criteria,  and  equally  important  to  transfer,  is  the  necessity  to  vary 
the  situations  and  possibilities  in  the  environment  that  the  learner  is  expected  to  generalize  the  learning. 

The  concepts  of  far  and  near  transfer  have  significant  ramifications  for  instructional  design  practitioners  since  the 
design  process  is  grounded  in  the  application  of  learning  both  immediately  after  education  or  training  or  in  the  future  as 
learners  continue  their  education  or  employment.  Far  and  near  transfer  are  similar  to  Gagne’s  definition  of  lateral  and 
vertical  transfer,  their  similarity  can  be  found  through  examining  the  functions  of  time,  learning  of  subordinate  skills  and 
content  complexity.  Near  transfer  is  concerned  with  application  of  instruction  similar  in  complexity  to  the  immediate 
future,  where  far  transfer  has  the  expectation  of  generalizing  or  applying  learning  over  a longer  time  frame,  and  often  in 
varying  situations  and  contexts  to  the  original  training.  A similarity  between  near  and  lateral  transfer  can  be  found  in 
their  expectation  of  applying  concepts  and  procedures  to  problems  or  situations  equal  in  complexity  to  those  practiced  in 
the  instruction.  Moreover,  both  vertical  and  far  transfer  have  the  expectation  that  the  learner  apply  their  learning  over 
time  and  to  new  concepts  and  problems  often  to  those  more  complex  and  unlike  those  presented  and  practiced  in  the 
original  instruction.  This  brings  the  discussion  to  the  internal  dimension  of  both  vertical  and  far  transfer.  Vertical  and 
far  transfer  rely  on  the  learner  having  mastery  of  a variety  of  knowledge,  information  and  skills  which  in  turn  enhances 
the  possibility  of  transfer  occurring.  The  practicing  instructional  designer,  through  their  design  of  the  instruction,  should 
build  in  a positive  environment  for  learning.  This  environment  should  not  only  allow,  but  also  strongly  encourage, 
learners  to  experience  "real  life"  situations  in  their  instruction  and  to  rehearse  their  perceptions  of  the  concepts  and 
information  with  other  learners  while  they  are  being  taught. 


226 


Although  most  researchers  and  theorists  in  the  domains  of  instructional  design,  educational  psychology  and 
education  support  the  concept  of  transfer,  there  are  those  who  question  the  underpinnings  of  the  transfer  theories  posited 
by  Gagne  and  others  of  like  mind.  Singley  and  Anderson  (1989)  have  written  an  entire  book  addressing  transfer  from  the 
perspective  of  enhancing  the  probability  of  it  occurring  in  the  area  of  cognitive  skills.  Although  they  have  devoted 
considerable  effort  to  their  investigation  of  transfer,  Singley  and  Anderson  question  some  of  the  premises  related  to 
vertical  transfer  and  the  effectiveness  of  hierarchical  analysis  and  the  identification  of  prerequisite  skills  as  a method  for 
enhancing  transfer  in  curriculum  design.  They  don’t  rule  out  the  possibility  of  the  success  of  this  method;  however,  they 
question  the  effectiveness  of  it  and  recommend  more  specific  research  in  this  area. 

Application  of  Gagne’s  transfer  theories  to  contemporary  instructional  design  practice  are  many,  but  few  are  any 
clearer  than  Dick  and  Carey’s  (1990)  discussion  of  goal  analysis  and  subordinate  skills  analysis.  They  suggest  that 
designers  can  easily  be  misled  by  focusing  on  what  learners  need  to  know  rather  than  what  the  learner  must  do. 
Furthermore,  they  insist  that  when  analyzing  subskills,  the  designer  must  ask  what  is  it  that  the  student  must  already 
know  how  to  do,  the  absence  of  which  would  make  it  impossible  to  learn  this  subordinate  skill?  The  utilization  of  this 
rhetorical  question  to  illustrate  a point  related  to  transfer  of  learning,  is  a clear  indication  of  the  impact  of  Gagne’s  work 
these  authors  and  subsequently  on  instructional  design  practice  given  that  it  is  found  in  what  was  described  earlier  as  a 
high  profile  and  widely  used  practitioner  text. 

Summary  and  Conclusions 

Attention  here  will  be  directed  toward  bringing  closure  to  the  discussion  of  the  lasting  influence  Gagne’s  theories 
have  had  on  practice.  Further  discussion  of  salient  research,  theories  and  practice  presented  in  the  paper  will  be  included. 

Perhaps  the  best  place  to  start  is  with  Gagne  (1989)  himself.  In  the  Preface  to  Studies  of  Learning:  50  Years  of 
Research.  Gagne  says: 

Learning  theory  has  maintained  its  interest  for  me  over  many  years.  However,  the  questions  addressed  in  my 

research  have  usually  been  practical  ones,  or  at  least  have  been  strongly  influenced  by  practical  considerations. 

(p.  6) 

This  statement  and  others  made  in  the  preface  of  his  book  reflect  Gagne’s  perception  of  his  efforts  to  use  research  and 
theory  to  solve  practical  problems.  As  Gagne  nears  the  end  of  the  preface,  in  what  appears  to  be  an  introspective 
comment  about  life  choices,  says:  "My  move  to  Florida  State  in  1969  was  the  beginning  of  a concentrated  effort  devoted 
to  teaching  and  writing  in  the  field  of  instructional  design"  (p.  6).  Gagne  seems  to  be  acknowledging  here  that  his 
intention  was  to  link  his  vast  research  and  theory  base  in  instruction  and  learning  with  learners  and  the  profession 
through  teaching  and  writing. 

Gagne’s  perception  of  his  life’s  vision  seems  to  be  one  of  research  and  writing  which  initially  focused  on 
learning,  and  eventually,  moved  toward  important  contributions  to  theory.  Gagne  has,  through  his  unwavering 
examination  of  the  practice  of  instruction  and  applying  it  to  how  human  beings  learn,  contributed  greatly  to  the  building 
of  a foundation  for  the  field  of  instructional  design.  Clearly  his  interest  in  learning  in  schools,  as  related  to  curriculum, 
and  the  examples  he  used  to  illustrate  his  theories  are  evidence  of  his  interest  in  applying  his  theories  to  practice.  His 
evolution  from  experimental  psychologist  to  an  instructional  and  learning  theorist,  whose  focus  became  one  of 
application  of  cognitive  theories  to  instructional  design,  is  indicative  of  not  only  his  flexibility,  but  also  his  interest  in 
instructional  design  practice.  His  place  in  the  history  of  instructional  design  practice  is  most  certainly  secure  from  both  a 
foundational  as  well  as  an  applicational  perspective. 

Concluding  from  the  examination  of  Gagne's  influence  on  curriculum  it  is  clear  that  his  work  has  been 
significant.  Evidence  of  his  influence  can  be  found  in  the  many  applications  of  his  theories  and  research  to  a wide  variety 
of  content  areas,  age  levels  and  learning  environments.  Additionally,  his  theories  have  withstood  the  test  of  time  having 
been  applied  to  curriculum  of  various  types  over  the  course  of  50  plus  years.  As  mentioned  earlier  his  influence  on  the 
curriculum  of  science  has  perhaps  been  most  broad  based,  long  lasting  and  nationally  acclaimed. 

Questions  remain  as  to  how  Gagne's  theories  will  endure  in  light  of  the  many  available  and  competing  theories 
that  practicing  instructional  designers,  curriculum  specialists  and  educators  now  have  as  options  when  designing 
instruction. 
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A partial  answer  might  be  found  by  revisiting  an  interview  Gagne  had  with  the  editor  of  Educational  Technology 
in  1982.  The  editor  asked  Gagne  if  he  thought  instructional  design  would  eventually  transition  entirely  from 
behaviorism  to  cognitive  psychology  or  would  there  remain  a behavioral  presence.  Gagne  responded  by  saying: 

I think  that  designers  who  work  with  cognitive  learning  theory  in  mind  really  incorporate  the  important  parts  of 

behavioral  theory.  Therefore,  I think  the  answer  to  your  question  must  be  "yes."  I do  believe  that  the  cognitive 

approach  will  come  to  dominate,  if  it  hasn't  already,  (p.  580) 

Does  Gagne's  response,  from  13  years  ago,  offer  us  any  clues  to  lasting  impact  of  his  theories  in  light  of  the 
large  theory  base  available  in  the  mid-90s?  The  answer  is:  a qualified  yes,  since  designers  are  often  pragmatists  in  their 
everyday  practice  of  instructional  design.  Subsequently,  they  will  select  those  theories  and  elements  of  theories,  that 
seem  logical  and  have  a high  probability  of  working  in  the  situations  and  environments  they  find  themselves. 

A final  comment  on  Gagne's  future  influence  on  curriculum  must  take  into  account  the  writers,  researchers  and 
theorists  in  curriculum  publications.  These  documents  would  lead  one  to  conclude  that  constructivism  will  be  the 
dominant  force  in  curriculum  construction  in  the  nineties.  Earlier,  when  discussing  curriculum,  it  was  noted  that  Gagne 
was  cited  only  once  in  the  1991-94  ASCD  Handbook  while  constructivism  and  situated  cognition  were  cited  often. 
Furthermore,  the  ASCD  publications  have  generous  citations,  methods,  and  corresponding  activities  that  are  very  situated 
or  constructivist  in  nature. 

Returning  to  Finiley's  work  may  offer  another  perspective  for  the  reader.  Finiley's  criticisms  of  Gagne  in  1983, 
which  were  fundamentally  philosophical,  might  have  been  harbingers  to  the  late  1980s  and  early  1990s.  One  thing 
seems  clear,  if  the  differences  between  constructivist  philosophy  and  the  inductive  empiricism  of  Gagne  are  perceived  to 
be  irreconcilable  by  a majority  of  the  theorists,  this  may,  eventually,  decrease  Gagne's  influence  on  instructional  design 
practice  and  curriculum  construction.  However,  another  scenario  may  be,  that  practitioners  will  utilize  Gagne's  theories 
more  selectively. 

The  genesis  of  Gagne's  theories  found  their  way  into  my  practice  before  I knew  they  existed  or  what  they  were, 
and  10  years  before  I heard  the  name  Robert  M.  Gagne.  I learned  from  the  instruction,  which  I believe  was  designed 
using  his  theories,  because  it  was  logical,  provided  me  with  enough  practice  to  reach  mastery  and  subsequently  be 
successful.  I taught  from  the  instructionally  designed  materials  which  utilized  his  theories  because  they  were 
comprehensive,  well  planned  and  worked.  I continue  to  utilize  his  theories,  selectively,  25  years  later,  as  a significant 
part  of  my  practice  for  many  of  the  same  reasons.  Gagne's  theories  will  continue  to  evolve  as  scholars  analyze  his  work 
in  search  for  new  meaning.  Gagne  has  a lasting  place  in  the  future  of  instructional  design  and  educational  practice.  His 
theories  and  positions  will  undoubtedly  be  reinterpreted,  modified  and  expanded,  but  his  prominence  will  remain. 
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Training  professionals  perform  in  a great  variety  of  roles  as  they  apply  their  competencies  to  the  human  resource 
development  challenges  facing  their  organizations  (McLagan,  1989).  Two  primary  duties  of  trainers  include  designing 
and  delivering  instruction.  Each  of  these  duties  is  becoming  more  challenging  as  technology  evolves.  The  use  of 
technology  in  training  has  grown  tremendously  over  the  past  five  years;  tools  have  improved  and  have  produced  several 
changes  in  the  way  training  is  being  designed  and  delivered  (Haag,  1993).  Contemporary  design  and  delivery  systems 
include  computer-based  training  systems,  multimedia  systems,  electronic  performance  support  systems  and 
telecommunication  systems  for  distance  learning.  In  addition,  computer  technology  is  being  used  to  enhance  traditional 
classroom  training. 

Although  not  every  organization  has  implemented  these  new  design  and  delivery  systems,  the  number  of 
organizations  using  these  systems  increases  each  year.  A recent  survey  of  organizations  with  more  than  100  employees 
indicated  that  48%  are  using  computer-based  training,  27%  are  using  multimedia  systems  and  43%  are  using  some  type 
of  distance  learning  system  to  deliver  training  (Industry  Report,  1995). 

The  use  of  technology-based  delivery  systems  in  the  training  piocess  has  many  potential  benefits  for 
organizations.  For  example,  the  use  of  computer-based  technology  in  the  design  and  delivery  of  training  can  result  in 
greater  learning  gains,  more  consistent  and  acceptable  job  performance,  enhanced  cost-effectiveness  and  greater  flexibility 
regarding  the  time  and  locations  of  training  (Hannum,  1990).  Several  research  studies  have  concluded  that,  under  the 
right  circumstances,  computer-based  delivery  systems  are  considerably  more  cost  effective  than  classroom  training  and 
produce  learning  that  is  at  least  equal  to  what  can  be  achieved  in  a classroom  (Haap.,  1993).  Technology  can  also  help 
when  addressing  the  needs  of  geographically-dispersed  trainees  and  in  reducing  the  need  for  classroom  facilities  (Perlstein, 
1993). 

However,  the  value  derived  from  the  use  of  technology  in  training  is  not  due  to  the  hardware  itself  but  rather  to 
the  instructional  processes  that  technology  can  support  (Hannum,  1990).  To  be  effective  in  their  positions,  today  s 
trainers  must  possess  competencies  needed  to  perform  in  an  increasingly  technological  environment.  They  must  have  a 
solid  understanding  of  learning  theories  and  methodologies  and  be  able  to  apply  this  knowledge  to  the  development  and 
delivery  of  training  using  computer-based  technologies,  distance  learning  systems  and  other  types  of  instructional 
technology  (Hannum,  1990).  Trainers  who  lack  these  skills  may  be  limiting  their  effectiveness  and  their  ability  to 

obtain  positions  or  advance  in  many  areas  of  the  field. 

Developing  and  maintaining  expertise  in  instructional  technology  can  be  a challenge  to  trainers  for  many 
reasons.  Because  instructional  technology  is  an  emerging  field,  many  of  the 

concepts  associated  with  this  field,  including  1 multimedia  or  distance  learning,  have  taken  on  a wide  range  of 
meanings,  resulting  in  confusion  for  practitioners  (Anglin,  1991).  Also,  because  many  of  these  technologies  are  still 
evolving,  there  are  few  standards  in  the  field.  The  hardware  and  software  associated  with  these  technologies  is  continually 
being  changed,  upgraded  or  replaced.  It  is  often  difficult  for  both  new  and  experienced  trainers  to  design,  develop  and 
implement  hardware  and  software  in  a timely  manner  when  the  technology  environment  is  dynamic  and  the  rate  of 
obsolescence  is  increasing  (Anglin,  1991).  Finally,  many  trainers  who  have  spent  several  years  using  traditional  training 
methods  and  media  may  resist  or  feel  uncomfortable  with  new  technology  (Schaaf,  1992). 

There  is  a need  for  a better  understanding  of  the  competencies  required  by  trainers  regarding  the  use  of  new 
technologies  in  training  and  how  these  competencies  may  be  developed.  Although  there  have  been  several  needs 
assessments  conducted  to  identify  the  competencies  of  trainers,  including  the  American  Society  for  Training  and 
Development's  Models  for  HRD  Practice  and  the  competency  studies  published  by  the  International  Board  of  Standards  for 
Training.  Performance  and  Instruction,  typically,  studies  of  this  type  do  not  provide  any  detailed  information  about  the 
competencies  needed  to  use  specific  types  of  hardware,  software  or  delivery  systems. 

Currently,  there  are  several  resources  that  could  provide  training  to  trainers  who  need  to  develop  their  knowledge, 
skills  and  abilities  in  designing  and  delivering  instruction  using  contemporary  instructional  technology.  These  resources 
include  undergraduate  and  graduate  courses  at  colleges  and  universities;  courses,  workshops  and  certificate  programs 
offered  by  trade  schools  or  technical  colleges;  and  conferences  and  seminars  offered  by  professional  associations 
(Lindstrom,  1 994).  Many  hardware  and  software  vendors  also  provide  training  to  organizations  that  purchase  their 
equipment  and  products.  Finally,  trainers  can  train  themselves  using  training  courseware,  computer  tutorials,  videotapes, 
books  and  manuals  (Lindstrom,  1994).  However,  because  these  programs.and  resources  are  relatively  new,  no  statistics 
exist  regarding  the  peicentage  of  trainers  who  complete  formal  or  self-paced  programs  or  how  successful  these  programs 
are  in  meeting  the  needs  of  trainers. 

In  addition,  even  trainers  who  are  highly  skilled  in  the  use  of  instructional  technology  may  experience  difficulty 
when  implementing  new  types  of  delivery  systems  in  the  work  environment.  There  are  often  barriers  in  organizations 
that  inhibit  the  successful  implementation  of  instructional  technology  in  training.  These  barriers  may  include  high 
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costs,  lack  of  management  support,  lack  of  trainer  skills,  cultural  defaults  for  the  classroom  experience  and  failure  to 
identify  needs  adequately  (Gery,  1994). 


Statement  ot  the  Problem 

Little  is  known  about  the  nature  of  the  challenges  encountered  by  trainers  as  they  attempt  to  incorporate 
computer-based  technologies  and  distance  learning  systems  into  their  training  processes  and  programs.  Few  studies  have 
examined  the  impact  of  these  technologies  on  the  role  of  the  trainer. 

Purpose  of  the  Study 

The  purpose  of  this  study  is  to  provide  current  information  on  the  implementation  of  instructional  technology 
in  employee  training  and  the  competencies  needed  by  trainers  to  utilize  instructional  technology  in  their  jobs.  This 
information  may  be  used  to  assist  training  professionals  in  determining  their  continuing  education  or  training  needs  in 
the  area  of  instructional  technology.  The  information  also  may  be  used  by  universities,  professional  organizations  and 
others  who  provide  degree  programs  in  human  resource  development  or  instructional  technology  in  developing  relevant 
curricula. 


Research  Questions 

This  study  sought  to  find  answers  to  the  following  questions: 

1 . What  types  of  computer-based  technologies  and  distance  learning  systems  are 
being  used  to  deliver  training  in  business  and  industry? 

2.  What  types  of  computer-based  technologies  and  distance  learning  systems  will  be  used  to  deliver 
training  in  the  future? 

3.  What  are  the  competencies  that  are  needed  for  trainers  to  deliver  instruction  using 
computer-based  technologies  and  distance  learning  systems? 

4.  Where  are  trainers  obtaining  the  competencies  that  are  needed  to  deliver 
instruction  using  computer-based  technologies  and  distance  learning  systems? 

5.  What  barriers  exist  in  the  workplace  that  prevent  trainers  from  using  computer- 
based  technologies  and  distance  learning  systems  to  deliver  training  in  the 
workplace? 


Significance  of  the  Study 

Technology  has  dramatically  changed  the  way  can  be  which  training  is  designed  and  delivered.  These 
technological  trends  necessitate  that  training  professionals  learn  new  job  skills  (Lindstrom,  1994).  This  study  was 
designed  to  benefit  training  professionals,  human  resource  development  managers,  academicians  and  other  training 
providers  by  providing  practical,  timely  information  that  may  be  used  to  update  trainer  skills  and  training  programs. 
This  study  will  provide  current  information  on  the  implementation  of  instructional  technology  in  training  efforts  at  a 
time  when  it  is  crucial  for  trainers  to  expand  their  repertoire  of  skills  in  this  area. 

In  order  to  best  prepare  training  professionals  with  the  necessary  knowledge,  skills  and  competencies,  training 
providers  need  to  have  an  accurate  picture  of  the  current  skill  requirements  (Morlan  & Lu,  1994).  The  findings  of  this 
study  may  be  used  by  universities  and  other  organizations  to  develop  and  revise  degree  programs,  courses,  seminars, 
workshops,  and  self-study  materials  to  meet  the  instructional  technology  training  needs  of  human  resource  development 
professionals. 


Methodology 

This  study  was  conducted  during  a six-month  period  from  June  to  December  in  1994.  First,  a literature  review 
focusing  on  the  use  of  instructional  technology  in  training  was  conducted.  Next,  a questionnaire  was  developed  by  the 
researcher  and  reviewed  by  a group  of  eight  trainers,  research  consultants  and  experts  in  the  field  of  instructional 
technology.  The  first  section  of  the  survey  was  to  contain  demographic  items,  including  job  title  of  respondent,  type  of 


organization  and  size  of  organization  where  the  respondent  is  employed.  In  the  second  section,  respondents  were  to 
identify  (a)  how  technology  is  currently  being  used  to  design  and  deliver  training  in  their  organizations,  (b)  their 
perceptions  of  the  types  of  technology  that  will  be  used  to  design  and  deliver  training  in  the  future,  (c)  the  level  of 
competency  required  of  trainers  in  each  type  of  technology,  and  (d)  sources  of  competency  development  each 
technology  In  the  third  section  of  the  survey,  respondents  were  asked  to  identify  barriers  in  the  workplace  which  limited 
the  implementation  of  instructional  technology  in  training. 

This  study  was  designed  to  determine  the  perceptions  of  training  professionals  regarding  the  use  of  technology  in 
the  design  and  delivery  of  instruction.  The  population  selected  for  this  study  included  members  of  the  National  Society 
for  Performance  and  Instruction,  specifically  those  members  living  and/or  working  in  the  following  Midwestern  states: 
Illinois,  Iowa,  Michigan,  Minnesota  and  Wisconsin.  This  group  was  selected  over  the  other  professional  training 
associations  because  of  its  focus  on  performance  technology.  The  association's  1993-94  national  membership  directory 
was  used  to  obtain  a list  of  members  of  the  population  for  this  study.  In  this  directory,  members  were  listed 
alphabetically  by  state.  In  the  five  Midwestern  states  included  in  the  study,  there  were  a total  of  1,093  members.  A 
systematic  sample  of  members  was  selected.  This  method  of  sampling  is  appropriate  when  a list  of  elements  is  available 
and  when  the  list  is  arranged  in  a manner  that  will  not  interfere  with  the  purpose  of  the  study  (Babbie,  1990).  This 
method  also  assured  that  a proportional  number  of  individuals  from  each  of  the  five  states  would  be  included  in  the 
sample. 

The  following  steps  were  taken  to  determine  the  size  of  the  sample.  First,  a decision  was  made  to  establish  a 
confidence  level  of  95%  (0.95)  for  the  results  of  the  study.  Then  the  formula  outlined  in  How  to  Determine  Appropriate 
Surrey  Sample  Size  (Narins,  1994)  was  applied  to  calculate  the  appropriate  sample  size  for  the  population.  For  a 
population  of  1,093,  a sample  of  381  was  required  to  produce  the  desired  confidence  level.  This  method  of  determining 
sample  size  was  generous  and  provided  latitude  against  typical  sources  of  error  including  non-response.  The  first  member 
was  chosen  at  random  from  the  list,  of  members  in  the  designated  five-state  area.  Then  every  third  member  was  selected 
until  the  sample  had  reached  the  desired  size. 

Instrument  Development 

Given  the  purpose  of  the  study,  the  research  questions  to  be  answered  and  the  size  of  the  sample,  a mail 
questionnaire  appeared  to  be  the  most  economical  and  appropriate  data  collection  technique.  The  instrument  was  developed 
through  a careful  examination  of  similar  studies  found  in  the  review  of  literature.  The  instrument  was  designed  to  collect 
data  as  a self-administered  questionnaire. 

The  instrument  consisted  of  an  1 1 x 17-inch  sheet  of  paper  which  was  printed  on  both  sides  and  folded  into  a 
four-page  booklet.  The  body  of  the  questionnaire  was  divided  into  three  sections.  In  the  first  section,  respondents  were 
asked  if  their  current  position  involved  designing,  delivering  or  managing  training.  At  this  point,  individuals  who  were 
not  currently  working  in  the  field  of  training  and  development  were  instructed  to  send  back  the  survey  without  answering 
the  remaining  survey  items.  Respondents  who  were  currently  employed  in  the  field  were  instructed  to  provide  other 
demographic  information,  including  their  job  title  and  the  size  and  type  of  their  organization,  and  to  complete  the 
remaining  three  pages  of  the  survey. 

The  second  and  third  pages  of  the  survey  contained  a grid  that  listed  32  types  of  instructional  technologies 
divided  into  categories.  These  categories  included  computer-based  training  systems,  multimedia  systems,  electronic 
performance  support  systems,  virtual  reality,  distance  learning  systems  and  computer  presentation  systems.  Additionally, 
five  areas  were  further  divided  into  subcategories.  In  each  category  and  subcategory,  respondents  were  asked  to  identify  if 
they  used  each  technology  in  their  training  efforts  and  if  they  planned  to  use  each  technology  in  the  next  three  years. 

They  were  also  provided  with  an  option  to  list  additional  technologies  which  were  not  included  on  the  questionnaire. 

In  this  same  section,  respondents  were  asked  to  assume  that  their  organization  planned  to  use  each  technology 
and  to  identify  the  levels  of  competency  that  would  be  needed  to  implement  the  technology.  The  levels  included  the 
ability  to  use  or  assist  trainees  in  the  use  of  a particular  technology,  the  ability  to  assess  the  effectiveness  of  a 
technology,  the  ability  to  select  a technology  for  an  organization  and  the  ability  to  develop  a program  or  system  using 
the  technology.  Respondents  were  allowed  to  select  as  many  levels  as  they  felt  were  appropriate. 

Respondents  were  then  asked  to  identify  where  they  obtained  or  would  plan  to  obtain  competency  in  each 
technology.  A list  of  training  sources,  including  colleges  and  universities,  technical  colleges,  seminars  and  conferences, 
vendor- sponsored  training  and  self-study  methods,  was  provided. 
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In  the  third  section,  printed  on  the  fourth  page  of  the  survey,  respondents  were  asked  to  identify  barriers  in  the 
workplace  which  limited  the  use  of  instructional  technology  in  training.  In  this  section,  respondents  were  provided  with 
a list  of  potential  barriers  and  were  asked  to  indicate  if  the  barriers  were  present  in  their  work  environment.  These  barriers 
included  insufficient  funding;  hardware  incompatibility;  lack  of  management  interest  or  support;  lack  of  time,  knowledge 
or  technical  skills  among  trainers;  lack  of  interest  among  trainees;  inadequate  needs  assessment;  and  lack  of  technical 
support.  Respondents  could  also  identify  additional  barriers  if  desired. 

A cover  letter  was  developed,  printed  and  mailed  with  each  survey.  The  cover  letter  explained  the  purpose  of  the 
study  and  the  format  of  the  questionnaire.  It  also  emphasized  the  importance  of  respondent  input  and  provided  the  name 
and  complete  address  of  the  researcher.  The  cover  letter  also  contained  the  informed  consent  information  required  by  the 
researchers  university. 

Pata-Cgligolion 

The  381  questionnaires  were  mailed  with  cover  letters  and  postage-paid  return  envelopes  on  October  7,  1994.  A 
pencil  was  included  as  incentive  for  individuals  to  respond  to  the  survey.  Seven  of  the  surveys  were  returned  to  the  sender 
due  to  an  incorrect  or  outdated  address.  These  individuals  were  removed  from  the  sample,  reducing  the  sample  size  to 
374.  Thirty  percent  of  the  sample,  1 12  individuals,  returned  the  survey  by  November  5,  1994.  On  November  6,  1994,  a 
second  mailing,  which  included  the  questionnaire,  a second  cover  letter,  a postage-paid  envelope,  and  a packet  of  instant 
coffee,  was  mailed  to  the  remainder  of  the  sample.  By  December  12,  1994,  49%  of  the  sample,  184  individuals,  had 
returned  the  survey.  A phone  follow-up  was  conducted  between  December  12  and  December  16,  1994.  An  attempt  was 
made  to  phone  every  individual  who  had  not  returned  the  survey.  During  the  phone  calls,  individuals  were  asked  several 
questions  related  to  the  use  of  instructional  technology  in  training  and  reminded  to  send  back  the  original  survey. 

As  a result  of  the  phone  interviews,  54  additional  people  were  removed  from  the  sample  because  they  were  no 
longer  employed  at  the  organization  listed  in  the  directory.  Thus,  the  final  sample  size  was  determined  to  be  320. 
Following  the  phone  calls,  another  15  surveys  were  returned.  The  final  number  of  responses  was  199,  for  a response  rate 
of  62%.  Of  the  199  individuals  who  returned  the  survey,  52  individuals  indicated  that  they  were  not  involved  in  the 
design  or  delivery  of  employee  training  programs  (mostly  university  faculty  members,  students  and  retirees)  and  did  not 
complete  all  of  the  items  on  the  survey.  This  left  a total  of  147  completed  questionnaires  to  be  analyzed. 

Data  Analysis 

On  December  23,  1994,  the  surveys  were  delivered  to  the  University  of  Wisconsin-Stout's  Academic  Computer 
Center  to  be  analyzed.  The  data  were  tabulated,  and  the  surveys  were  returned  to  the  researcher  on  February  3,  1995.  In 
all  sections  of  the  survey,  each  item  was  analyzed  in  terms  of  frequency  of  each  response  and  overall  percentage  for  each 
option  provided.  In  addition,  Z-tests  on  the  difference  of  proportions  between  the  "yes"  responses  of  current  users  and  the 
"yes"  responses  of  those  who  planned  to  use  each  technology  wdre  conducted  to  determine  if  there  were  significant 
differences  between  current  and  planned  future  usage.  Chi-square  tests  were  conducted  to  determine  if  there  were 
significant  differences  between  the  various  training  sources  selected  by  respondents. 

Results 

The  results  of  the  survey  indicated  that  organizations  are  currently  using  a wide  range  of  technologies,  and  there 
are  12  technologies  that  are  currently  being  used  by  at  least  50%  of  the  respondents.  These  technologies  include 
computer-based  training,  computer  tutorials,  computer  simulations,  computer  presentation  systems,  presentation 
software,  electronic  performance  support  systems,  on-line  help  systems,  information  databases,  multimedia  systems, 
LCD  panels,  LCD  video/data  projectors  and  local  area  networks. 

These  were  the  same  technologies  that  50%  or  more  of  the  respondents  indicated  they  plan  to  use  in  the  next 
three  ye°u.  However,  respondents  indicated  that  they  plan  to  make  significantly  greater  use  of  multimedia  development 
and  delivery  tools  including  authoring  programs,  interactive  video,  CD-ROM,  compact  disk  interactive  and  digital  video 
interactive.  They  also  indicate  that  they  plan  to  use  more  complex  technologies  for  their  computer-based  training  and 
electronic  performance  support  systems,  including  hypertext,  expert  systems,  embedded/concurrent  training,  intelligent 
tutoring  and  virtual  reality.  There  will  also  be  greater  use  of  computer  conferencing  to  deliver  training  over  distances. 

According  to  respondents,  there  will  be  less  use  of  certain  types  of  distance  learning  systems,  such  as 
audioconferencing  and  one-way  video.  This  would  be  logical  as  technology  continues  to  evolve  and  provides  more 
advanced,  interactive  systems  for  the  delivery  of  distance  education  (Gery,  1994).  There  will  also  be  a decrease  in  the  use 
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of  computer  presentation  systems  to  deliver  classroom  training.  This  decline  will  occur  as  companies  abandon  classroom 
training  for  more  effective  and  cost  efficient  electronic  instruction  delivered  at  the  desktop  (Galagari,  1994). 

The  levels  of  competency  required  to  implement  instructional  technology  in  training  programs  were  consistent 
across  27  of  the  32  types  of  technologies  included  in  the  study.  Overall,  respondents  reported  that  the  ability  to  use  or 
assist  trainees  in  the  use  of  the  technology  was  the  most  highly  needed  competency.  The  ability  to  evaluate  the 
effectiveness  of  a specific  technology  was  also  frequently  identified.  The  ability  to  develop  programs  or  systems  was 
identified  by  few  respondents  as  being  necessary  for  most  technologies. 

Vendor-sponsored  training  and  self-study  methods  proved  to  be  the  most  popular  choices  for  developing 
competency  in  instructional  technology.  Attending  seminars,  conferences  or  workshops  was  frequently  identified  for 
developing  competency  in  some  technologies;  however,  it  was  not  the  primary  method  for  competency  development  in 
any  specific  technology.  Significantly  fewer  individuals  indicated  that  they  would  attend  courses  and  programs  at 
universities,  four-year  colleges  or  technical  colleges  to  develop  their  skills  in  any  of  the  areas  included  in  the 
questionnaire. 

The  respondents  indicated  that  a lack  of  time  and  a lack  of  financial  resources  are  the  major  barriers  in 
implementing  instructional  technology  in  training  efforts,  as  these  barriers  were  cited  by  approximately  75%  of  the 
respondents.  Lack  of  compatibility  between  systems,  lack  of  management  support,  lack  of  technical  support  and  lack  of 
trainer  skills  are  barriers  that  were  identified  by  more  than  50%  of  the  respondents.  These  findings  were  consistent  with 
the  literature  on  this  topic,  which  indicates  there  are  several  reasons  why  technology  has  not  been  fully  integrated  into 
training  programs,  including  high  costs,  lack  of  management  support  and  lack  of  skills  among  trainers  (Gery,  1994). 

Conclusions 

From  the  findings  of  this  study,  it  can  be  concluded  that  the  major  types  of  instructional  technologies  used  in 
training  and  development  will  not  change  dramatically  over  the  next  three  years.  However,  there  are  several  newer,  more 
sophisticated  technologies  that  will  be  used  with  greater  frequency  in  the  future.  The  respondents'  current  and  planned 
uses  of  instructional  technology  are  consistent  with  other  recent  studies  on  this  topic  that  indicate  there  will  be  a greater 
use  of  interactive  technologies  that  will  change  how,  when,  and  where  trainees  learn  (American  Society  for  Training  and 
Development,  1994).  It  is  predicted  that  in  the  future  more  companies  will  utilize  digital  multimedia  technologies  and 
individualized  performance  support  systems  to  provide  flexible  training  opportunities  to  workers  (Galagan,  1994). 

From  the  survey  responses,  it  can  be  concluded  that  it  is  far  more  important  for  trainers  to  be  able  to  use  and 
evaluate  new  technologies  than  to  be  able  to  design  and  develop  their  own  programs  or  systems.  The  data  from  the 
survey  support  the  concepts  found  in  the  literature  in  this  area.  Past  studies  on  this  topic  have  concluded  that  trainers 
should  be  familiar  with  the  applications  of  instructional  technology;  however,  program  or  system  development  is 
generally  done  by  computer  programmers  or  media  specialists  with  expertise  in  these  areas  rather  than  by  trainers 
themselves  (Spitzer,  1988). 

It  was  determined  by  the  respondents  that  vendor-sponsored  training  and  self-study  methods  are  the  primary 
sources  of  competency  development  in  instructional  technology.  Seminars,  conferences  and  other  training  programs 
sponsored  by  professional  organizations  appear  to  play  a lesser  role,  and  universities  and  technical  colleges  appear  to  play 
a minimal  role  in  providing  trainers  with  knowledge  and  skills  in  computer-based  training,  multimedia  systems,  EPSS, 
distance  learning  systems  or  computer  presentation  systems. 

Finally,  it  can  be  concluded  that  a lack  of  time  and  a lack  of  financial  resources  are  the  major  barriers  to 
implementing  instructional  technology  in  training  efforts.  Lack  of  compatibility  between  systems,  lack  of  management 
support,  lack  of  technical  support  and  lack  of  trainer  skills  are  also  significant  barriers.  However,  there  does  not  appear 
to  be  a lack  of  trainee  interest  in  using  instructional  technologies  or  a general  lack  of  support  for  training  efforts. 

Recommendations 

This  study  was  designed  to  assist  training  professionals,  human  resource  development  managers,  academicians 
and  others  who  offer  training  and  degree  programs  by  providing  information  on  how  instructional  technology  is  currently 
being  used  in  training  and  how  it  may  be  used  in  the  future.  These  various  groups  may  find  the  results  of  this  study 
useful  in  future  planning  efforts. 
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Recommendations  for  Practitioners 

It  is  recommended  that  training  professionals  and  their  managers  use  the  data  generated  by  this  study  in  defining 
current  and  future  training  needs  and  in  identifying  resources  to  obtain  new  skills  and  competencies  in  instructional 
technology.  In  particular,  training  professionals  should  become  familiar  with  the  technologies  that  are  currently  being 
used  by  more  than  50%  of  the  respondents'  organizations.  Training  professionals,  whether  working  in  small,  mid-sized, 
or  large  organizations,  should  also  become  knowledgeable  regarding  digital  technologies  and  other  emerging  technologies 
that  large  numbers  of  organizations  plan  to  use  in  the  future.  Skills  in  using  computer  systems  and  electronic  support 
systems  have  been  formally  recognized  as  essential  competencies  for  training  professionals  (McLagan,  1989).  The  shift 
from  face-to-face  training  to  delivering  information  during  the  performance  of  work  will  require  all  trainers  to  become 
familiar  with  numerous  delivery  technologies  (Galagan,  1994). 

The  data  from  this  study  suggest  that  training  professionals  should  focus  on  developing  competency  in  the  use 
and  evaluation  of  the  various  technologies  included  in  the  survey.  However,  in  a small  number  of  organizations,  it  is 
also  required  that  trainers  select  and  develop  programs  and  systems.  In  addition  to  learning  about  hardware  and  software, 
comments  from  respondents  and  the  literature  indicate  that  trainers  also  should  possess  competencies  in  using  traditional 
media  technologies.  Trainers  should  also  understand  the  process  of  applying  appropriate  instructional  technology  to 
performance  problems  (Piskurich,  1993). 

Recommendations  for  Training  Providers 

The  results  of  this  study  may  also  assist  faculty  in  colleges,  universities  and  technical  colleges;  directors  of 
professional  organizations;  hardware  and  software  vendors  and  others  who  provide  courses,  programs  and  training  in  the 
field  of  instructional  technology.  The  majority  of  respondents  in  this  study  preferred  vendor-sponsored  training  and  self- 
study  methods  to  meet  their  instructional  technology  training  needs.  However,  several  respondents  indicated  that  they  and 
their  colleagues  have  not  been  trained  in  many  aspects  of  instructional  technology  and  lack  the  knowledge  or  skills 
necessary  to  be  effective  in  this  area.  Therefore,  it  is  recommended  that  vendors  and  organizations  that  provide  self-study 
materials  expand  their  offerings  in  instructional  technology  training,  particularly  in  the  area  of  emerging  technologies  to 
help  address  this  unmet  training  need. 

Although  several  post-secondary  institutions  offer  programs  in  instructional  technology,  few  trainers  seem  to 
be  taking  advantage  of  these  offerings.  It  is  recommended  that  undergraduate  and  graduate  programs  in  training  and 
development  and  instructional  technology  review  their  instructional  technology  courses  and  competencies  in  light  of  the 
findings  of  this  study  to  ensure  that  their  courses  are  relevant  and  appropriate  for  training  professionals. 

Due  to  inadequate  budgets  for  equipment  and  laboratories,  it  is  often  difficult  for  colleges  and  universities  to  stay 
current  with  technology  (Lindstrom,  1994).  However,  if  universities  and  technical  colleges  are  truly  interested  in 
meeting  the  instructional  technology  training  needs  of  training  professionals,  it  is  recommended  that  they  form 
partnerships  with  vendors  or  professional  associations  to  assist  in  offsetting  the  costs  of  providing  hardware  and  software 
training  on  college  campuses. 
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INTRODUCTION 


A major  United  States  automotive  manufacturing  company  invested  extensively  developing  a new  computer- 
based,  Computer  Numerical  Controlled  (CNC)  machining,  robotics  and  high  technology  facility.  The  work  environment 
and  job  content  had  changed,  BUT  the  employees  hadn't.  They  were  totally  unprepared  for  the  new  job  requirements 
regardless  of  management's  communications  and  worker  involvement  in  organizing  and  equipping  the  facility.  The 
employees  were  not  fully  competent  to  operate  the  equipment,  activate  and  utilize  the  computers,  analyze  and  diagnose 
problem  causes  or  make  timely  repairs.  Action  was  needed...immediately ! 

THE  PLAN 

A joint  salaried  and  hourly  employee  task  force  formulated  a plan  to  address  both  the  immediate  and  long-term 
concerns.  Their  recommendation  was  to  create  an  in-plant  Lifelong  Learning  Center  for  the  21st  Century  to  provide  for 
personal  renewal  and  technical  updating  of  the  workforce.  Company  and  union  leadership  fully  endorsed  the  plan  and 
pledged  their  continued  support.  A training/education  consultant  was  hired  to  assure  needed  confidentiality  which  seemed 
to  be  a real  concern  for  the  hourly  employees. 

COMMUNICATION 

Five  statewide  schools  and  three  consultant  firms  provided  extensive  communications  about  their  respective  personal 
renewal  and  work  oriented  programs.  The  workforce  was  informed  via  all-employee  meetings,  videotapes,  written 
bulletins/announcements,  live  intra-plant  video  transmissions,  in-plant  computer  inter-net  systems,  electronic  bulletin 
boards  and  union  membership  meetings.  Following  this  communications  blitz,  a learning  needs  assessment  and  interest 
questionnaire  was  developed,  communicat-ed,  and  hand  delivered  in-plant  to  all  2,182  employees.  The  questionnaire  was 
on  the  inter-net  computer  system,  loo,  so  its  completion  could  be  done  and  transmitted  electronically.  Each  employee 
was  offered  a one-on-one  confidential  meeting  if  he/she  desired.  This  was  especially  critical  for  those  lacking  reading 
and/or  writing  skills. 

The  1,831  (84%)  questionnaires  completed  showed  287  employees  expressed  interest  in  pursuing  either  a high 
school  or  college  degree  program.  Basic  work  oriented  remedial  courses  were  requested  by  179  employees  and  they  all 
indicated  a desire  for  specialized  technical  courses,  too.  A full  range  of  basic  through  in-depth  technical  courses  were 
selected  by  388  employees  while  over  1,300  employees  selected  specific  training  programs  such  as  interpersonal 
relations,  conflict  resolution,  listening,  problem  solving,  effective  meetings,  etcetera. 

Follow-up  information  meetings  were  held  concerning  the  (l)high  school  degree,  (2)  associates  degree,  (3) 
bachelors  degree,  and  (4)  masters  degree  programs.  Those  employees  wanting  work  oriented  remedial  education  were 
individually  counseled  and  completed  a computerized  assessment.  Help  was  provided  to  those  needing  assistance 
especially  in  the  reading  area. 

Training  program  offerings  were  consolidated  and  prioritized  according  to  technological  and  manufacturing  needs. 
Those  critical  competencies  for  the  new  high-tech  facility  received  top  priority  to  satisfy  the  immediate  needs.  A master 
schedule  was  developed  and  employees  needing  the  high  technology  and  computer  skills  were  trained  immediately  while 
the  remainder  of  the  workforce  wanting  this  training  were  invited  to  sign-up  for  future  offerings.  A master  sign-up  time 
was  announced  for  all  the  various  training  programs  and  courses  and  the  employees  responded  beyond  expectations. 

THE  BEGINNING 

The  personal  renewal,  technical  updating  Lifelong  Learning  Center  for  the  21st  Century  began  and  experienced 
immediate  and  significant  success.  During  the  first  year,  thirty-eight  (38)  employees  took  courses  toward  their  high 
school  degree  or  GED  (General  Equivalency  Degree).  Several  of  these  courses  of  study  were  offered  via  television. 
Another  1 18  employees  began  pursuing  various  college  degrees  all  within  the  facility.  The  teaching  methodology 
utilized  was  many  faceted:  the  normal  teacher-student  inter-action,  the  televised  Indiana  Higher  Education 
Telecommunications  System  (IHETS)  which  broadcast  courses  from  the  statewide  universities,  computerized  independent 
study  and  selected  independent  study  coupled  with  mentoring.  The  college  degree  based  students  were  about  equally 
divided  between  associate  degrees  from  Purdue  University  School  of  Technology  or  Indiana  Vocational  Technology 
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College,  bachelor  degrees  from  Ball  State  University  School  of  Technology  or  Business  School,  and  masters  degrees 
from  Ball  State  University  or  Purdue  University  in  either  executive  development  or  engineering,  respectively. 

All  courses  other  than  laboratory  or  experiential  based  courses  were  offered  via  IHETS,  as  available,  and  several  courses 
used  inter-active  computer  based  courses  for  credit  through  Penn  State  University  (International  Correspondence  Schools). 
Seventy-three  (73)  employees  enrolled  in  work  oriented  remedial  courses  such  as  arithmetic,  reading,  writing,  blueprint 
reading,  measuring  and  metrics.  Much  of  their  training/education  was  via  small  group  interaction  facilitated  by  a 
professional  instructor,  interactive  computer  systems  and  video  programs.  Individualized  help  was  readily  available  for 
those  needing  it.  Over  400  employees  completed  technical  oriented  courses/programs  such  as  computer  applications. 
Computer  Numerical  Control  (CNC)  programming,  CNC  machine  set-up  and  operation,  basic  and  advanced  electronics, 
mechanics  and  hydraulics/pneumatics  concurrent  with  machine  and  robot  troubleshooting,  diagnosis  and  repair.  Every 
conceivable  method  of  training  and  education  was  utilized  from  computers,  to  laser  discs,  to  simulation  models,  to 
IHETS  etcetera.  In  addition  to  these  first  year  accomplishments,  over  1 ,000  employees  participated  in  other  training 
programs  many  of  which  utilized  CBT,  interactive  computer  systems  and  IHETS.  The  first  year  was  a resounding 
success  and  the  subsequent  years  have  been  equally  as  good.  Goodnight  - 3 
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The  purpose  of  this  study  was  to  explore  the  way  music  operated  in  the  mental  processing  of  computer- 
supported  instructional  messages,  whose  other  modes  were  text  and  graphics.  The  experiment  examined  the 
effect  of  music  on  the  immediate  recall  and  understanding  of  three  equivalent  science  lessons  delivered  by  a 
computer.  The  objective  properties  of  the  music  were  manipulated  to  produce  feelings  that  were  congruent  with 
the  psychological  dynamics  of  the  lessons'  instructional  strategies.  This  technique  sought  to  enhance 
achievement  by  stimulating  multiple  (triple)  encoding  in  short  term  memory.  It  aligned  an  abstract  musical 
input,  which  elaborated  subjectively  on  content,  with  instructional  strategy,  which  is  a subjective  lesson 
element.  This  research,  therefore,  proposed  and  tested  a variable,  labeled  MISI  (Music-Instructional  Strategy- 
Integration),  that  moderated  the  instructional  treatments.  The  study  also  sought  subjects'  affective  evaluations  of 
the  lessons  accompanied  by  music  as  well  as  their  preferences  for  music  in  association  with  academic  endeavors. 

The  experimental  procedure  examined  the  effects  on  achievement  of  three  forms  of  ninth  grade  lessons 
on  physical  science  topics,  each  of  which  was  differently  moderated  by  music  or  ran  in  silence.  This  process 
involved  repeated  measures  of  subjects,  taken  at  random  from  classes  of  ninth  grade  students.  The  investigation 

employed  a Graeco-Latin  Square,  type  GLSFF-3  design.  This  was  a fractional  factorial  design,  that  evaluated 
two  crossed  treatments  in  the  context  of  three  groups  of  subjects. 

The  primary  analysis  of  variance  showed  no  significant  statistical  results  regarding  achievement.  There 
was  no  statistically  significant  difference  in  the  subjects  ratings  of  the  musical  patterns,  and  no  relationship 
between  achievement  scores  and  subjects'  ratings  was  found.  The  subjects  were  consistent  in  favoring  an 
association  of  music  with  academic  activity. 

This  study  found  that  the  inclusion  of  music  can  stimulate  positive  student  affect  toward  computer- 
supported  instruction  and  toward  science  instruction.  It  identifies  the  need  for  additional  research  with  multiple 
encoding,  and  with  literal  music,  which  depicts  content  by  becoming  one  of  its  objective  parts. 


Computer-based  multimedia  is  rapidly  becoming  an  important  instructional  tool  (Heinich,  Molenda,  & Russell, 
1993;  Schwier  & Misanchuk,  1993).  Research  results  document  positive  learning  outcomes  associated  with  the  proper 
instructional  use  of  this  media  format  (Kozma,  1991).  Interactive  multimedia  programs  deliver  instructional  messages 
through  a mix  of  text  and  graphics,  still  and  motion  images,  voice,  sound  effects,  and  music.  These  complex  messages 
are  the  means  through  which  interactive  multimedia  presentations  gain  their  considerable  power  to  effectively 
communicate  with  learners  (Heinich,  Molenda,  & Russell,  1993). 

Little  research  exists  on  the  use  of  music  to  enhance  learning  in  the  context  of  interactive  multimedia.  This 
void  is  not  consistent  with  the  universal  recognition  of  music  as  one  of  the  most  powerful  forms  of  human  expression. 
Music  is  celebrated  worldwide  for  its  profound  ability  to  create  meaning  (Abraham,  1980).  Neither  is  this  situation 
consistent  with  the  steadily  improving  quality  of  digitized  sound  (McKell  & McKell,  1994).  Most  of  the  research  on  the 
instructional  use  of  music  in  older  multimedia  formats  has  concentrated  on  its  affective  influence  (the  feelings  it  induced) 
when  the  music  was  used  as  a background  context  for  teaching.  These  research  findings  are  mixed  and  inconclusive 
(McIntyre  & Cowell,  1984;  Ebisutani  & others,  1991;  Jaspers,  1991). 

Music  is  regularly  listed  as  one  of  the  standard  communication  modes  for  constructing  interactive  multimedia 
presentations  (e.g.,  Heinich,  Molenda,  & Russell,  1993),  despite  the  absence  of  research-supported  guidelines.  When 
guidance  is  offered  about  how  to  insert  music  into  this  instructional  format,  it  typically  is  general  and  highly  subjective 
(e.g.,  Burger,  1993).  Schwier  and  Misanchuk  (1993)  advise  that  music  can  be  used  to  "augment”  the  presentation  and  to 
lend  "positive  emotional  texture”  (p.  268).  The  details  of  the  process  are  left  to  the  intuition  of  program  designers  and 
authors.  One  cause  for  this  shortcoming  is  that  a grounded,  universally  accepted  understanding  of  the  way  music  operates 
in  the  mental  processing  of  multi-modal  messages  sent  by  a multimedia  system  has  not  been  developed  (Kellaris,  Cox, 

& Cox,  1993). 

Research  has  documented  a positive  impact  on  learning  resulting  from  additive,  dual  encoding  when  linguistic 
inputs  (text/speech)  are  combined  with  illustrations  (Fleming  & Levie,  1993).  Rayburn  and  Tyson  (1982)  extended 
additive  encoding  to  a third  level  when  they  concluded  that  music  could  compliment  other  images  (linguistic  and  visual) 
and  provide  additional  information  to  the  brain  in  the  form  of  auditory  images.  There  has  not,  however,  been  any 
reported  research  into  the  use  of  triple  encoding  with  music. 

Hannafin  and  Hooper  (1993)  provide  an  excellent  summary  of  the  state  of  the  art  for  constructing  multi-modal 
messages.  They  maintain  that: 
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Learning  via  multi-modal  instruction  improves  when  there  is  sufficient  conceptual  and  temporal  overlap  between 

the  information  presented  in  each  modality  (p.  196). 

They  anchor  the  above  principle  of  learning  in  Paivio's  (1971)  finding  concerning  the  effectiveness  of  dual  encoding. 
Hannafin  and  Hooper  state  that  dual  encoding  depends  on  the  additive  effect  of  encoding  mechanisms  in  short-term 
memory. 

The  example  given  is  that  of  combining  text  with  illustrations,  wherein  the  relevant  encoding  mechanisms  are 
separate  but  complementary.  The  text  is  said  to  be  a linguistic  code  form,  which  is  normally  (but  not  necessarily) 
scanned  horizontally.  To  read  text  is  a visual  event,  which  means  that  it  is  a spacial  phenomenon.  Most  people, 
however,  employ  a serial  protocol  in  dealing  with  text.  The  encoding  of  illustrations  is  different.  While  it  too  is  visual, 
and  therefore  also  a spacial  event,  the  illustrations  are  consumed  in  parallel.  An  image  code  form  and  another  variety  of 
encoder  are  involved  in  the  latter  case. 

Hannafin  and  Hooper  (1993)  identify  two  conditions  governing  the  effectiveness  of  multi-modal  messages  in 
learning.  The  first  is  that  multi-modal  messages  should  present  information  to  students  in  code  forms  that  are  congruent. 
This  is  consistent  with  the  findings  of  Kellaris,  Cox,  and  Cox  (1993)  for  persuasive  messages  in  advertising  and  also 
with  the  findings  of  Boltz,  Schulkind,  and  Kantra  (1991)  for  schematic  processing.  The  text  and  illustrations  of  Hannafin 
and  Hooper's  example  must  be  redundant,  focusing  on  the  same  teaching  points.  These  authors  warn  that  non-redundancy 
may  interfere  with  additive  encoding. 

The  second  condition  is  a requirement  that  each  presentation  mode  within  these  complex  messages  appeal  to 
sepa-ate  encoding  mechanisms.  To  appeal  to  different  encoders  in  the  student's  working  memory,  separate  code  forms 
(e.g.,  text  and  an  illustration)  must  be  used  to  enhance  the  formation  of  meaning.  As  a counter  example,  Hannafin  and 
Hooper  (1993)  caution  against  an  identical  presentation  of  words  via  both  text  and  speech. 

Previous  interference  studies  have  shown  that  human  beings  are  capable  of  using  several  different  types  of 
encoding  mechanisms  (Ashcraft,  1989;  Leahey  & Harris,  1985).  Not  only  are  there  linguistic  (text/speech)  and  image 
encoders,  but  there  are  sonic,  olfactory,  kinesiologic,  and  possibly  other  combined  types  of  encoders. 

It  is  unknown  whether  encoding  mechanisms  combine  operations  to  achieve  richer  forms  of  encoding  or  whether 
they  simply  operate  in  parallel  to  achieve  additive  encoding.  Although  their  pattern  of  operation  is  not  yet  clear, 
encoding  mechanisms  seem  to  exist  in  the  limited  capacity  of  short-term  memory.  Studies  in  Dual  Task  Analysis  infer 
some  limit  to  the  positive  effect  of  additive  encoding  (Ashcraft,  1989). 

Schwier  and  Misanchuk  (1993)  conceive  multi-modal  instructional  messages  as  consisting  of  core, 
supplemental,  and  peripheral  elements.  Music  is  usually  inserted  as  a peripheral  element  in  the  mix  (Kellaris,  1991),  in 
part  because  no  objective  means  of  relating  it  to  the  other  messages  elements  has  been  developed.  Music  therefore  tends 
to  be  relegated  to  an  opening,  closing  or  background  for  the  program.  This  conception  and  its  inherent  design  convention 
seem  short-sighted,  and  they  preempt  music's  full  potential  to  contribute  to  the  presentation. 

The  nature  and  artistic  tradition  of  music  impedes  its  use  as  a tool  of  technology.  Standard  practice  for  dealing 
with  music  in  education  is  much  more  subjective  than  objective.  The  educational  use  of  music  tends  to  be  mainly 
intuitive  for  the  following  reasons.  First,  the  power  of  music  to  create  meaning  is  thought  to  operate  primarily  through 
emotional  pathways  (Schwier  & Misanchuk,  1993;  Burger,  1993).  Second,  music  appreciation  has  typically  been 
approached  historically  and/or  stylistically,  in  terms  of  extant  compositions.  Finally,  the  objective  use  of  music  requires 
some  facility  with  theory,  which  is  complex  and  difficult.  Music  has  a special  vocabulary  and  mathematical  precision. 

The  problem  with  this  appropriation  of  music  is  that  it  fosters  an  incomplete  and  unscientific  approach.  It 
suggests  a use  of  music  that  is  based  only  on  musical  genres  (e.g.,  a march  versus  a dirge)  and  only  by  association  (e.g., 
the  deep  "Dum  ...  Ba-Dum"  from  the  movie,  JAWS,  suggests  extreme,  unseen  jeopardy). 

This  tendency  to  appropriate  music  only  on  the  basis  of  intuition  obscures  a more  technological  approach  to  its 
use.  Music  can  be  selected  and  inserted  into  instructional  programs  more  precisely  if  that  insertion  is  based  on  its 
objective  properties.  There  is  empirical  evidence  that  music  can  be  manipulated,  using  its  objective  properties  like 
tempo,  tonality,  and  texture  to  produce  psychological  feelings  such  as  arousal,  surprise,  and  pleasantness  (Kellaris  & 
Kent,  1993).  Instructional  designers,  while  paying  some  attention  in  the  past  to  tempo  (e.g.,  Wakschlag,  Dietz,  & 
Zillman,  1982),  do  not  yet  take  full  advantage  of  this  way  of  applying  music  to  their  programs. 

Previous  reviews  of  the  literature  have  been  narrowly  focused.  Research  is  being  conducted  across  a number  of 
disciplines  that  are  important  in  understanding  the  effects  of  music  within  instructional  environments.  Also,  much  of  the 
existing  literature  is  contradictory.  Numerous  studies  have  used  music  as  a variable.  The  most  recent  studies  are  finally 
focusing  on  distinct  properties  of  music. 
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Review  of  the  Literature 


This  analysis  of  the  literature  did  not  locate  any  reports  of  research  into  the  use  of  music  as  an  instructional 
message  element  in  the  context  of  interactive  multimedia.  Some  textbooks  mention  this  application  of  music  (e.g., 
Schwier  & Misanchuk,  1993),  but  their  coverage  is  limited. 

There  is,  however,  a substantial  amount  of  literature  about  the  use  of  music  in  teaching  with  other  media.  Most 
of  this  material  exists  either  in  the  form  of  programmatic  essays  in  magazines  and  journals  or  as  research  reports.  The 
program  guides  start  with  the  assumption  that  music  can  enhance  learning.  They  generally  endorse  the  use  of  music  in 
education,  describe  its  application  in  a particular  instructional  situation,  and  disclose  positive  affective  responses  by 
students. 

The  research  literature  reports  mixed  findings  about  the  effects  of  music  on  instruction.  Dr.  Dolf  Zillman 
(1994),  a noted  scholar  in  the  field  of  communications,  interprets  the  consensus  of  the  literature  as  follows: 

The  evidence  concerning  the  use  of  music  in  educational  efforts  is  most  discouraging.  Surely,  as  we  have 
shown,  music  can  help  to  attract  children  to  the  educational  message  (in  selected  situations).  But  once  they  are 
exposed,  the  presence  of  music  is  detrimental  to  learning.  That's  what  the  experimental  research  tells  us.  The 
findings  are  very  consistent,  showing  that  it  is  sweet  illusion  for  educators  to  think  that  music  could  further  the 
learning  process.  Music  is  a message  that  competes  with  educational  information  for  attention,  and  it  usually 
wins  the  contest. 

There  is,  however,  a minority  body  of  evidence  together  with  an  even  greater  amount  of  teacher  intuition  that 
opposes  Dr.  Zillman's  judgment.  This  subset  of  the  literature  supports  the  counter  argument  that  music  can  enhance  the 
presentation  of  educational  information. 

Such  differences  of  opinion  can  only  mean  that  a full  understanding  of  the  dynamic  interaction  between  music 
and  instruction  has  not  yet  been  achieved.  That  conclusion  constitutes  the  need  for  further  study. 

Musical  Acuity  / Intelligence  / Academic  Achievement 

The  general  connection  between  musical  acuity  and  intelligence  or  academic  achievement  has  been  the  focus  of  a 
number  of  studies.  Cary’s  (1987)  review  of  the  pertinent  research  indicated  a definite  connection  between  the  exposure  of 
children  to  music  and  higher  levels  of  intelligence.  This  applied  also  to  cognitive  skills  such  as  problem  solving,  risk 
taking,  and  creativity.  Dryden  (1992)  demonstrated  this  linkage  in  fifth  grade  students.  She  documented  a positive 
correlation  between  students'  instruction  in  music  and  the  development  of  verbal  skills.  Hilliard  (1993)  amassed  five  case 
studies  profiling  the  characteristics  of  creative  adolescents.  All  loved  music. 

A connection  between  music  and  academics  has  been  identified.  The  exact  nature  of  the  relationship  has  not, 
however,  been  determined. 

Musical  Expression 

The  expressive  power  of  music  has  been  known  since  ancient  times.  Martin  Luther,  for  example,  had  the 
highest  praise  for  music.  Luther  ranked  it  second  only  to  theology  in  his  hierarchy  of  God's  gifts  (Randal,  1986).  Other 
teachers,  through  the  ages,  have  used  music  in  various  ways  to  make  students  more  - and  sometimes  less  - receptive  to 
learning. 

How  music  operates  to  express  meaning  remains  a major  concern  of  aesthetic  criticism.  Expressionists  base 
their  understanding  in  the  composer's  intentions  when  creating  a work  or  in  the  appreciator's  feelings  when  consuming 
the  work.  Formalists  argue  that  musical  expression  derives  from  the  listener’s  cognitive  recognition  of  the  musical 
elements  of  the  composition  melding  into  a whole.  Moderate  perspectives  urge  a synthesis  of  these  affective  and  rational 
responses  (Behrend,  1989). 

Burger  (1993)  provides  guidance  for  designers  about  the  use  of  music  in  multimedia.  He  categorizes  music  as 
literal  or  abstract.  Literal  music  depicts  content  by  becoming  one  of  its  objective  parts,  i.e.,  a sound  effect  such  as  birds 
chirping  or  water  rushing.  Abstract  music  elaborates  subjectively  on  content.  It  operates  as  a communications  device, 
using  emotions  (e.g.,  sadness,  love,  joy,  patriotism,  etc.)  to  embellish  the  content.  Literal  music  operates  on  or  off 
screen,  conveying  proximity  or  environment.  Abstract  music  operates  off  screen,  stimulating  mood  and  a sense  of  time, 
geographic  location,  association,  signature  and  continuity,  and  memorability.  Burger  also  introduces  elements  - melody, 
harmony,  rhythm,  instrumentation,  musical  genres,  time  signature,  tempo,  structure,  and  silence  - that  can  be  used  to 
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convey  meaning.  He  states  that  these  devices  "can  imply  much  more  than  what  is  being  said  and/or  portrayed  visually" 

(p.  334). 

The  scholarly  concern  with  such  aesthetic  expression  extends  back  to  the  1800's  (e.g.,  Gurney,  1880;  Gundlach, 
1935;  Rigg,  1937;  Watson,  1942;  Holbrook  & Bertges,  1981).  Gurney  explained  this  expressive  power  of  music  as 
early  as  1880  in  the  following  way: 

(T}he  power  of  music  to  suggest  external  objects  and  events  and  intellectual  conceptions  ...  may  take  place  in 
two  ways.  First,  the  actual  sounds  and  motions  of  the  music  may  perceptibly  resemble  actual  sounds  and 
motion  of  other  things  ...  The  second  way  in  which  images  of  external  facts  may  be  suggested  by  music  is  by 
general  qualities  ...  {T}he  same  calm  and  steady  musical  flow  which  might  suggest  a quiet  succession  of  waves 
has  naturally  an  expression  of  tranquillity  {sic}  corresponding  to  the  same  idea  (p.  349-50). 

A good  example  of  classical  music's  ability  to  produce  sound  effects  is  found  in  Grofe's  Grand  Canyon  .Suite. 

One  hears  in  this  work  a credible  musical  image  of  a donkey's  "clopping"  along  a canyon  trail  and  also  a vivid 
representation  of  a thunder  storm.  There  are  similar  musical  dynamics  in  Prokofiev  s Peter  and  the  Wolf  and  in  Saint 
Saens'  Carnival  of  the  Animals. 

Good  examples  of  classical  music's  power  to  express  the  content's  general  qualities  are  Wagner's  Tristan  UJld 
Isolde  of  love  and  passion;  Smetana's  Ma  Vlast  of  patriotism;  Holst's  The  Planets  (Mars)  of  militarism;  Berlioz's 
Requiem  f?.nd  Movement  - Dies  Irael  of  fear;  Stravinsky's  Rite  of  Spring  of  sexuality;  Mahler’s  Resurrection  Symphony 
of  transcendence;  Beethoven's  Ninth  Symphony  of  joy;  Verdi's  Stabat  Mater  of  lament,  etc.  Other  types  of  music  are 
equally  capable  of  such  expression. 

Human  Response 

Humans  respond  both  physically  and  mentally  to  music.  The  physical  response  has  been  documented  in  studies 
of  efforts  to  combat  text  anxiety.  Such  efforts  have  produced  mixed  results.  These  experiments  usually  measure  and 
report  ?h<*.  impact  which  music  can  have  on  pulse  rate,  blood  pressure,  skin  chemistry,  etc.  (e.g.,  Blanchard,  1979; 
Moseley,  1990). 

How  humans  respond  mentally  to  the  stimulus  properties  of  music  is  varied  and  complex.  There  are  both 
affective  and  cognitive  responses  to  music,  which  are  conditioned  by  individual  differences  (Rayburn  & Tyson,  1982). 
Irish  (1993)  found  that  when  adults  have  free  choice  of  words  to  respond  to  music,  their  affective  and  cognitive  responses 
are  personal  and  idiosyncratic.  There  were,  however,  consistent  patterns  of  response  in  and  between  individuals  with 
similar  education  and  experience.  Personality  traits  also  affect  reactions  to  music  (Lewis,  1991). 

Paris's  (1993)  recent  study  is  but  one  example  of  the  powerful  impact  of  music  in  the  affective  domain.  She 
found  significant  results  when  assessing  the  effects  on  fifth  graders  of  stimulating  music  as  compared  to  sedative  music. 
Such  effects  can  be  time  sensitive.  Brentar  an  associates  (1994)  found  an  inverted  U-shaped  relationship  between 
frequency  of  exposure  to  rock  and  popular  songs  and  the  affective  response  of  undergraduates  toward  the  songs. 

The  traditional  view  has  favored  an  affective  response  (feelings,  emotions)  over  a cognitive  response  (empirical 
factual  knowledge  processing  and  problem  solving)  to  music  (e.g.,  Schwier  & Misanchuk,  1993).  Hylnka  (1980), 
writing  about  the  favorable  implications  of  general  film  music  for  television  and  other  instructional  films,  drew  the 
conclusion  that  only  affective  learning  is  influenced  by  music.  Recognizing  the  difficulty  of  dealing  experimentally  with 
aesthetics,  Price  (1986)  proposed  a precise,  professional  vocabulary  on  affective  responses  in  the  field  of  music. 

There  is  evidence  of  music's  ability  to  affect  cognitive  processes.  Bickel  (1992)  presented  a means  of 
investigating  the  cognitive  component  of  music  listeners’  responses  through  analysis  of  their  verbal  reports.  She  found 
that  seventh  and  eighth  graders  often  presented  creative  metaphors,  described  unique  perspectives  on  the  listening 
experience,  displayed  personal  idiosyncrasies,  found  overall  unifiers,  and  invented  unique  listening  tools. 

The  most  publicized,  contemporary  work  on  the  impact  of  music  directly  on  cognitive  operations  is  that  of 
Frances  Rauscher  (1993).  She  reported  finding,  in  two,  similar  experiments  with  different  audiences,  that  exposure  to 
music  enhances  specific  spatial  reasoning  skills,  without  necessarily  invoking  emotions. 

Dr.  Rauscher's  musical  treatment  (Mozart  - a sonata  for  pianos)  was  administered  to  the  sample  prior  to  the 
reasoning  task,  not  concomitantly  with  it.  Her  data  showed  that  this  musical  score  acted,  physiologically,  to  prime  the 
same  neural  net  that  was  involved  in  the  problem  solving  exercise.  She  found  also  that  this  beneficial  effect  of  music 
was  not  permanent.  It  dissipated  within  fifteen  minutes. 
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Other  information  concerning  music  and  the  brain  is  contained  in  a database  (Music  and  Brain),  which  is 
accessible  over  the  Internet.  This  database  offers  references  to  scholarly  articles,  books,  conference  proceedings,  theses, 
reprints  on  music  history,  education,  psychology,  and  the  physiology  of  hearing  and  making  music. 

There  is  evidence  that  students,  whose  mental  operations  favor  the  right  hemisphere  of  the  brain,  are  more 
appreciative  of  music.  Zalanawski  (1990)  analyzed  this  tendency.  She  reported  results  from  a mixed  sample  of 
undergraduates,  who  responded  both  verbally  and  vistully  to  a musical  selection.  Those  with  a right  brain  orientation 
showed  greater  appreciation  as  measured  by  attention,  understanding,  and  enjoyment  scores. 

It  is,  however,  an  oversimplification  to  portray  aesthetic  appreciation  and  creativity  solely  in  the  right 
hemisphere  and  rationality  only  on  the  left  side.  Levy,  as  early  as  1983,  concluded  that: 

The  two  hemispheres  differ  in  their  perceptual  roles,  but  both  sides  are  involved  in  the  creation  and  appreciation 

of  art  and  music.  Both  hemispheres  are  involved  in  thinking,  logic,  and  reasoning.  The  right  hemisphere  seems 

to  play  a special  role  in  emoiion,  e.g.,  humor.  If  students  are  emotionally  engaged,  both  sides  of  the  brain  will 

participate  in  the  educational  process  regardless  of  the  subject  matter,  (p.  68). 

There  is  additional  support  for  the  notion  that  both  hemispheres  of  the  brain  are  engaged  by  music  (MacKinnon,  1981 ; 
Merrion,  1988;  Piro,  1989). 

There  has  been  consideration  of  the  connection  between  music  and  thinking.  Marek  (1993),  using  music  to 
induce  moods,  found  no  significant  effect  on  memory.  Stratton  and  Zalanowski  (1985)  got  mixed  results  when  they 
tested  paired  associate  memory  with  imagery  and  repetition  instructions  with  and  without  background  music. 

Instructional  Uses 

Studies  concerned  with  the  effect  of  music  on  student  achievement  have  typically  examined  specific  settings, 
student  populations,  and/or  subject  matter.  These  often  overlap.  These  studies  have  generally  assessed  student  behavior 
when  different  types  of  music  were  presented  to  students  during  instruction.  The  range  of  types  of  music  used  with 
instruction  has  been  very  wide.  It  includes  rock,  classical,  easy-listening,  and  has  even  been  extended  to  rap  (Anderson, 
1993). 

SslriAg 

Teachers  have  long  been  concerned  with  the  creation  of  a setting  conducive  to  learning.  A holistic  approach  to 
teaching  means  appealing  simultaneously  to  the  student's  physical,  emotional,  and  cognitive  domains.  The  relatively 
recent  discoveries  about  brain  functioning  have  encouraged  this  approach. 

A program  of  this  type  is  Lozanov's  "Suggestive  Accelerated  Learning  Technique"  (SALT).  To  foster  holistic 
learning,  SALT  uses  music  only  as  a general  context.  The  music  is  selected  by  genres  and  applied  as  sonic  mass 
(Kellaris,  Cox,  & Cox,  1993)  simply  to  envelop  the  instruction.  Its  purpose  is  to  set  a mood,  induce  relaxation,  and  to 
be  broadly  suggestive.  The  hope  is  that  music  will  operate  as  a catalyst  for  learning  because  it  increases  receptivity  in 
the  brain. 

SALT  gained  a substantial  following  and  has  been  the  subject  of  considerable  experimentation  (Schuster  & 
Gritton,  1986).  There  is  a sizable  body  of  literature  pertaining  to  this  method,  much  of  which  deals  with  music.  The 
pertinent  experimental  results  with  SALT,  however,  are  mixed  and  inconclusive.  For  example,  Cullen  (1986)  reports 
significant  gains  from  the  use  of  accelerated  learning  in  language  instruction.  Conversely,  Confer-Owens  (1992)  found 
that  SALT  made  no  difference  in  teaching  study  skills  to  under-prepared  college  freshmen.  Moon  (1985)  used  a meta- 
analysis to  show  that  relaxation  techniques  had  only  a small  positive  effect  on  cognitive  academic  variables. 

There  are  other  studies  of  music  as  a favorable  context  for  academic  productivity.  A typical  example  is 
Kellogg's  (1982)  finding  that  many  successful  technical  writers  regularly  work  with  background  music.  He  surveyed  one 
hundred  twenty  seven  engineering  professors  about  their  academic  productivity  and  writing  habits.  Multiple  regression 
analysis  revealed  that  background  music  was  among  the  tools  used.  While  this  study  does  not  deal  with  instruction,  it 
may  have  implications  for  the  use  of  music  in  non-traditional  learning  environments. 

Student  Type 

Studies  have  been  done  with  specific  populations  of  learners.  The  major  groupings  have  been  of  regular 
students  versus  those  with  special  needs.  Regular  students  have  been  further  subdivided  according  to  age/grade  level  (e.g., 


Davidson  & Powell,  1986);  by  sex  and  perceptual  type  (e.g.,  Rayburn  & Tyson,  1982);  and  by  geography  (e.g.,  Eaton, 
1985;  Talabi,  1986;  Hicks,  1987). 

A typical  experiment  with  normal  students  was  run  by  Davidson  and  Powell  (1986).  They  observed  twenty  six 
students  in  the  fifth  grade  over  a four  month  period  to  determine  the  effect  of  easy-listening  background  music  on  on-task- 
performance  (O-T-P).  Time  series  analysis  indicated  a significant  increase  in  O-T-P  for  male  subjects  and  for  the  total 
class. 

Music  has  been  used  in  a wide  variety  of  special  education  situations.  The  relevant  literature  is  a blend  of  . 
programmatic  guides  and  reports  of  experiments.  The  former  are  generally  favorable  while  the  latter  report  mixed 
experimental  results.  Studies  done  with  this  population  typically  used  a background  music  treatment  and  measured 
variables  such  as  stress  levels,  hyper-activity,  anxiety,  psychomotor  skills,  and  cognitive  achievement. 

An  example  of  one  such  study  was  reported  by  Kleckley  (1989).  He  used  two  different  populations,  learning 
disabled  high  school  students  and  secondary  remedial  students.  The  study  involved  a three  week  unit  on  the  Constitution 
which  was  followed  by  four  parallel  quizzes.  Background  music  was  provided  during  alternate  testing  sessions.  Kleckley 
found  significant  variations  for  the  learning  disabled  group  in  the  form  of  lower  stress  levels  and  higher  test  scores  with 
background  music. 

Subject 

Studies  using  music  during  instruction  have  been  done  with  a wide  range  of  subject  matter,  again  with  mixed 
results.  Music  has  been  widely  used  in  teaching  language.  Studies  have  also  included:  mathematics,  spelling,  writing, 
vocabulary,  reading,  social  studies,  history,  and  multicultural  education.  This  group  of  studies  have  generally  measured 
achievement  and/or  affect. 

There  are  more  references  to  the  use  of  music  in  teaching  language  than  any  other  subject.  Several  experiments 
connect  music  to  verbal  learning.  Peters  (1993)  observed  that  music  seemed  to  encourage  verbal  auditory  learning  among 
second  graders.  Pietrick  (1988)  found  no  significant  changes  in  attitudes  of  sixth  graders  regarding  the  effects  of  music 
and  art  in  remedial  language  arts  development. 

Music  has  been  used  in  teaching  several  science  subjects.  Beauchamp  (1989)  included  a musical  factor  in  his 
comparison  of  the  effects  of  slide  plus  tape  presentations  in  biology.  He  found  significant  differences  among  four 
treatment  groups  in  the  degree  of  affective  response  and  cognitive  achievement.  Among  his  conclusions  was  that  music  - 
more  so  than  visuals  - prompted  long  range  positive  affective  responses. 

Smith  and  Davidson  (1991)  reported  a study  of  the  effects  of  music  listening  on  student  achievement.  The 
subjects  were  seventh-graders,  who  were  independently  learning  astronomy.  There  were  no  significant  achievement 
differences  among  students  who  learned  while  listening  to  rock,  classical,  easy-listening,  or  no  music  at  all. 

Literature  Reviews  Within  Education 

There  have  been  a number  of  literature  reviews  regarding  the  use  of  music  for  instructional  purposes  since  1984. 
Two  of  these  pertain  to  particular  groups,  which  are  exceptional  and  at-risk  students.  A third  is  more  general.  All  of 
these  reviews  conclude  that  the  research  results  which  they  considered  are  mixed  and  inconclusive.  Each  cites  the  need  for 
further  investigation. 

McIntyre  and  Cowell  (1984)  reviewed  the  research  concerning  the  effects  of  music  on  academic  performance  and 
behaviors  of  exceptional  students.  They  found  that  such  studies  pertained  primarily  to  achievement  in  mathematics, 
reading,  and  the  ability  to  attend  to  study  w iterials.  McIntyre  and  Cowell  also  discovered  that  the  research  concerning 
behaviors  of  exceptional  students  concentrated  on  task-related  behavior,  unacceptable  behavior,  interpersonal  conflicts, 
motor  activity  rate,  relaxation,  and  attention  span.  They  concluded  that  the  findings  regarding  the  effects  of  music  on 
studying,  mathematics,  reading  performance,  activity  rate,  and  social  behavior  are  unclear  and  often  contradictory.  They 
call  for  more  research  in  this  area. 

A second  literature  review,  performed  by  Ebisutani  and  associates  (1991),  focused  on  at-risk  students.  These 
reviewers  considered  the  effect  of  music  on  the  reading,  oral  language,  and  writing  abilities  of  marginal  students.  The 
intention  of  the  analysis  was  to  identify  and  describe  how  music  should  be  used  in  classrooms  to  facilitate  language 
development.  Ebisutani  et  al.  considered  three  types  of  publications,  theoretical  works,  research  reports,  and  essays  of 
practical  classroom  applications.  They  drew  two  conclusions.  First,  they  decided  that  the  research  results  were  mixed  and 
inconclusive.  The  evidence  did  not  suggest  to  them  that  music  has  the  potential  for  producing  a positive  effect  on 
reading  rate  and  writing  flue.xy.  Second,  they  found  that  the  theories  used  to  justify  the  use  of  music  in  reading  and 
language  arts  activities  are  not  firmly  grounded  in  any  research.  Accordingly,  these  investigators  joined  in  the  call  for 
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further  study,  pressing  for  examination  of  the  critical  variables  involved  in  music's  effects  on  the  literary  development  of 
students. 

A review  of  the  literature  performed  by  Jaspers  (1991)  deals  with  music  as  a component  of  instructional 
materials.  Observing  the  scarcity  of  literature  on  the  use  of  music  in  audiovisual  productions  and  its  implications  for 
their  design,  this  review  concentrates  on  the  relationship  between  music  and  emotion.  It  also  links  music  and  cognition, 
but  to  a lesser  extent. 

Jaspers'  (1991)  study  considered  music  to  be  a form  of  nonverbal  communication  (cf.  Hylnka,  1980).  His 
review  discusses  the  learning  outcomes  of  music  in  the  affective  and  cognitive  domains,  including  its  emotional  and 
structural  aspects.  Jaspers  concludes  that: 

{TJhere  appears  to  be  no  absolute  need  for  music  as  a component  in  an  instructional  programme.  Music  is, 
however,  one  out  of  a set  of  effective  means  to  evoke  a desired  emotional  state  in  the  viewer/listener,  or  to 
comment  on  information  conveyed,  or  to  line  out  certain  aspects  of  the  structure  of  the  problem  (p.  51). 

He  goes  on  to  suggest  that: 

by  manipulating  musical  variables  such  as  tempo,  modality,  {and}  rhythm,  the  designer  of  the  audio-visual 
programme  may  try  to  reach  more  emotional  ends,  and  by  manipulating  the  easy  to  recognize  musical  elements 
such  as  melody  and  instrumentation,  he  can  manipulate  the  more  cognitive  and  structural  ends  (p.  51). 

No  evidence  has  been  found  that  indicates  experimentation  by  Jaspers  to  test  this  conclusion. 

Other  Europeans  are  reported  to  be  studying  the  educational  use  of  music  including  Brosius  of  the  University  of 
Mainz  in  Germany  and  Broeckmann  at  the  University  of  Klagenfurt  in  Austria  (Zillman,  1994).  In  a German  article 
addressing  the  musical  dimension  of  education,  Dietrich  and  Wermelskirchen  (1990)  explore  how  musical  events  educate 
by  examining  the  debate  between  emotional  and  cognitive  responses  to  music. 

The  evidence  and  opinion  to  date,  concerning  the  use  of  music  to  enhance  educational  efforts  can  only  be  seen  as 
inconclusive  and  discouraging.  The  presence  of  music  is  thought  by  some  to  be  non-essential  in  instructional  programs, 
as  was  concluded  by  Jaspers.  Others  are  even  more  critical,  pointing  out  that  some  applications  of  music  can  be 
disruptive,  (e.g.,  Hannafin  & Hooper,  1993).  Such  interpretations  are  based  on  the  well  documented  distraction  effect  of 
music.  These  studies  begin  with  Fendrick's  work  (1937)  and  continue  until  recently,  (e.g.,  Zuk  & Danner,  1986). 

The  studies  reported  on  to  this  point,  have  one  thing  in  common.  They  have  generally  used  music  only  as 
background,  comparing  instructional  treatments  with  background  music  to  instructional  treatments  without  music.  The 
only  way  that  some  studies  attempted  to  manipulate  the  music  was  by  the  type  of  music,  thereby  altering  the  tempo, 
rhythm,  and/or  harmony  which  can  alter  moods  or  affective  responses.  One  additional  variation  to  the  use  of  music  in 
these  instructional  studies  has  been  where  background  music  has  accompanied  test-taking. 

Interdisciplinary  Studies 

Research  on  the  use  of  music  to  enhance  instruction  continues.  Significant  work  is  also  being  done  outside  the 
discipline  of  education.  The  relationship  of  music  to  various  settings,  audience  categories  and  behaviors,  and  varieties  of 
subject  matter  is  still  being  studied.  There  is  a new  concern,  however,  with  the  mechanics  of  communication, 
cognition,  and  the  objective  properties  of  music.  Hylnka  (1980)  suggested  a theoretical  basis  and  some  practical 
guidelines  for  using  music  in  multimedia  productions,  based  on  a critical  review  of  the  behavioral  research  on  film 
music.  His  theoretical  perspective  was  non-verbal  and  multiple  channel  communication. 

Seidman  (1981)  studied  the  role  of  music  in  motion  pictures  to  suggest  ways  that  educators  might  use  music  to 
improve  the  effectiveness  of  their  instructional  films.  Seidman  reviewed  several  studies  showing  that  tempo,  modality, 
rhythm,  and  harmony  interact  with  audience  characteristics  to  alter  moods  and  thus  shape  their  response  to  presentations. 
Seidman  was  not  optimistic  that  musical  accompaniment  to  media  productions  would  improve  student  performance  on 
achievement  tests  or  enhance  learning  and  retention  of  cognitive  content.  He  did,  however,  suggest  that  both  affective 
and  cognitive  interpretations  of  these  productions  are  influenced  by  such  "sonic  scenery"  which  the  musical  score  creates 
(p.  51). 

Rayburn  and  Tyson  (1982)  investigated  the  effect  of  background  music  with  lecture  tapes,  filmstrips,  and  films 
in  teaching  freshman  psychology  concepts.  They  compared  media,  visual  or  non- visual  (haptic),  and  male  and  female 
learners.  Treatment  conditions  included  music  during  presentation,  during  both  presentation  and  testing,  or  no  music. 
The  administration  of  the  treatments  - three  music  conditions  crossed  with  three  media  types  in  the  context  of  three 
groups,  constituted  a Graeco-Latin  Square,  type  GLSFF  3 design.  Results  using  difference  scores  by  sex  showed  no 
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effect  except  when  film  was  the  medium  of  instruction.  Film  was  a better  medium  of  instruction  for  women.  Results 
using  difference  scores  by  perceptual  type  showed  no  effect  except  when  lecture  tape  was  the  medium  of  instruction. 

With  the  music  at  presentation  condition,  visuals  improved  but  haptics  did  not.  Both  visuals  and  non-visuals  improved 
more  when  background  music  was  present  not  only  during  the  instruction  but  continued  during  the  test.  Rayburn  and 
Tyson  drew  the  conclusion  that,  "music  may  serve  as  an  auditory  image,  which  provides  additional  information  to  the 
brain  just  as  Paivio's  visual  images  seem  to  do"  (p.  21). 

Rayburn  and  Tyson's  (1982)  hypotheses  were  rooted  in  the  distraction  effect  of  music  and  in  the  widely  held 
assumption  that  males  are  more  visual  than  females.  They  predicted  that  when  music  was  added  to  media  presentations 
with  both  visual  and  verbal  elements,  the  visually  oriented  subjects,  i.e.,  males,  would  score  lower.  Conversely,  the 
haptically  oriented  subjects,  i.e.,  females,  were  expected  to  score  higher.  They  also  anticipated  that  the  same  background 
music  added  to  a lecture  presentation  would  lower  the  scores  of  the  visuals  and  would  not  affect  the  scores  of  the  haptics. 
These  hypotheses  were  not  supported  by  the  data.  The  test  scores  of  both  visuals  and  haptics  improved  when  the  music 
was  continued  through  the  test.  The  insertion  of  the  music  in  this  research  was  imprecise  (only  background  music  was 
used).  Also,  neither  the  media  type  nor  the  lesson  content  were  equivalent  which  are  necessary  when  using  their  chosen 
research  design. 

Wakshlag,  Reitz,  and  Zillman  (1982)  manipulated  the  tempo  of  music  in  an  educational  television  program  to 
study  its  impact  on  attention  gaining  and  information  acquisition.  Their  findings  offer  a mixed  endorsement  of  music. 
What  is  of  importance  is  their  experimental  technique  of  manipulating  the  musical  input  according  to  one  of  its  objective 
properties.  Later,  Reitz  (1984)  found  that  background  music’s  major  value  as  an  attentional  device  in  educational 
television  appears  to  be  its  ability  to  bring  viewers  back  to  the  screen  once  their  gaze  has  drifted  away. 

Boltz,  Schulkind,  & Kantra  (1991)  assessed  the  effects  of  background  music  on  the  remembering  of  filmed 
events  in  order  to  better  understand  schematic  processing,  a cognitive  process.  They  employed  a factorial  design,  in 
which  sixty  students  in  an  introductory  psychology  course  were  randomly  assigned  to  two  groups.  The  subjects  were 
presented  with  a series  of  filmed  episodes  in  which  the  mood  of  the  background  music  was  either  congruent  or 
incongruent  with  the  episode's  outcome.  For  one  group,  the  music  always  foreshadowed  the  ending  of  an  episode.  For 
the  other,  the  music  always  accompanied  the  final  outcome.  A control  group  saw  the  same  filmed  episodes  with  no 
music.  They  found  that  recall  was  significantly  enhanced  when  the  mood  of  accompanying  music  was  congruent  with 
the  mood  of  the  scene's  outcome.  They  also  found  that  mood-incongruent  music  was  more  effective  (memorable)  for  the 
foreshadowing  treatment.  Data  from  a cued-recognition  task  showed  that  music  could  provide  effective  retrieval  cues  for 
recognizing  the  scene  it  had  originally  accompanied.  In  addition,  music  with  positive  affect  consistently  produced  higher 
levels  of  performance  than  did  negative  ones. 

Kellaris  (1991),  Kellaris,  Cox  and  Cox  (1993),  and  Kellaris  and  Kent  (1993)  have  conducted  a multi-dimensional 
investigation  into  the  effect  of  background  music  on  ad  processing.  Kellaris  (199 1 ) and  Kellaris,  Cox  and  Cox  (1993), 
proposed  and  tested  a contingency  variable,  entitled  "Music-Message-Congruency."  This  construct  refers  to,  "the 
congruency  of  meanings  communicated  non-verbally  by  music  and  verbally  by  the  ad  copy".  The  experimental  results 
indicate  that,  "increasing  audience  attention  to  music  enhances  message  reception  when  the  music  evokes  message- 
congruent  versus  incongruent  thoughts"  (Kellaris,  Cox  & Cox,  1993,  p.51). 

Kellaris  and  Kent  (1993)  performed  an  exploratory  investigation  of  responses  elicited  by  music  varying  in 
tempo,  tonality,  and  texture.  Three  dimensions  of  human  feelings  - arousal,  surprise,  and  pleasantness  - emerged  from 
the  responses  measured.  They  reported  their  findings  as  follows: 

Pleasure  was  influenced  by  the  interactions  of  both  tempo  and  tonality  with  texture.  Faster  speeds  and  more 
consonant  keys  increased  the  pleasantness  of  classical,  but  not  Pop,  music.  Arousal  was  influenced  by  the 
interaction  of  tempo  with  texture.  Faster  speeds  produced  greater  arousal  among  subjects  exposed  to  Pop,  but 
not  classical,  music.  Feelings  of  surprise  were  influenced  by  tonality  such  that  less  consonant  keys,  i.e.,  atonal 
relative  to  minor,  minor  relative  to  major,  were  rated  more  surprising. 

The  stimulus  property  of  music  to  create  the  feeling  of  arousal  also  has  advocates  as  an  accelerated  learning 
technique  with  hyperactive  and  disabled  students  (Applegate  & Hamm,  1985;  Windwer,  1981 ). 

Analysis  of  the  literature  confirms  the  suspicion  that  there  is  as  yet  no  grounded,  universally  accepted 
understanding  of  the  way  music  operates  in  the  mental  processing  of  multi-modal  instructional  messages.  Furthermore, 
there  is  little  evidence  of  new  research  into  the  issue  as  it  regards  interactive  multimedia  instruction.  Past  evidence  about 
the  impact  of  music  on  learning,  in  any  format,  is  inadequate  and  inconclusive.  The  older  research  findings  are  certainly 
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mixed.  Consequently,  many  interpret  the  data  in  an  entirely  negative  way.  They  underscore  the  powerful  distracting 
properties  of  music,  and  they  favor  a minimal  use  of  music  in  educational  programs. 

The  practical  advice  which  is  often  given  to  designers  concerning  the  use  of  music  in  their  instructional  products 
is  meager.  It  sometimes  carries  a warning  that  the  wrong  kind  of  music  can  sabotage  the  lesson.  The  best  practical 
advice  about  the  most  effective  use  of  music  seems  to  exist  in  the  discipline  of  advertising. 

The  human  response  to  the  stimulus  properties  of  music  is  complex  and  difficult  to  understand.  That  process 
continues  to  be  the  source  of  debate  about  the  balance  of  affective  versus  cognitive  operations. 

The  question  about  a more  effective  use  of  music  in  educational  communications  continues  to  be  asked  and  the 
body  of  literature  on  the  issue  is  growing.  Research  is  being  conducted  internationally  and  in  academic  disciplines 
beyond  education,  e.g.,  psychology  (brain  research),  communications  (film  studies),  and  in  advertising.  Among  the 
growing  number  of  reports  about  positive  effects  of  music  on  learning,  cognition,  or  affective  response,  are  at  least  nine 
works  (Hylnka,  1980;  Seidman,  1981;  Wakshlag,  Reitz,  & Zillman,  1982;  Reitz,  1984;  Kellaris,  1991;  Boltz, 

Schulkind,  & Kantra,  1991;  Jaspers,  1991;  Kellaris,  Cox,  & Cox,  1993;  Kellaris  & Kent,  1993).  Such  work  is 
launching  a new  trend  in  the  way  music  is  applied  in  instructional  design.  Each  of  these  works  involves  inserting  music 
into  the  instruction  or  message  according  to  its  objective  properties. 

Studies  are  beginning  to  deal  with  the  integration  of  music  into  the  mechanics  of  communications  messages. 
This  technique  breaks  with  the  past  practice  of  using  background  music  throughout  the  lesson  and  expecting  it  to 
function  as  a contextual  catalyst  for  learning.  It  is,  therefore,  becoming  popular  to  use  elements  of  music  in  experiments 
as  moderating  variables,  which  embellish  the  treatment  variables.  This  practice  has  shown  some  positive  results 
(Kellaris,  Cox,  & Cox,  1993). 

The  present  study  was  designed  to  assess  a procedure  for  integrating  music  into  multimedia  instruction  to 
enhance  learning.  This  procedure  had  three  components.  First,  it  employed  multi-modal  messages,  which  attempted  to 
foster  triple  encoding  in  short  term  memory.  These  multi-modal  messages  were  constructed  of  three  congruent  code 
forms.  Text,  illustrations,  and  music  were  the  codes  that  were  used  because  each  is  processed  in  a different  encoder  in 
short  term  memory.  Second,  the  procedure  managed  the  musical  input  using  the  objective  properties  of  music.  Third, 
the  procedure  linked  the  musical  input  to  instructional  strategy  as  the  means  of  achieving  redundancy  among  the  code 
forms. 

This  research  proposed  and  tested  a variable,  labeled  Music-Instructional  Strategy -Integration  (MISI),  that 
moderated  the  instructional  treatments.  The  study  anticipated  that  the  beneficial  impact  of  music  in  multimedia 
instruction  is  directly  related  to  the  extent  that  the  music  enhances  psychological  operation  of  the  lessons'  instructional 
strategies. 

The  intention  of  MISI  was  to  systematically  leverage  music’s  ability  to  elicit  feelings  of  arousal,  surprise,  and 
pleasantness  through  manipulation  of  its  objective  properties  of  tempo,  tonality,  and  texture  (Kellaris  & Kent,  1993). 
MISI  linked  this  ability  of  music  to  two  standard  instructional  strategies  that  evoke  these  same  feelings.  The  operational 
strategies  were  affective  orientation  and  cognitive  dissonance.  The  first  involves  arousal,  and  the  second  entails  surprise 
and  pleasantness  when  the  dissonance  is  reconciled  (Hannafm  & Hooper,  1993).  These  two  strategies  guided  construction 
of  the  text  and  graphics  portions  of  the  treatment  lessons. 

Three  physical  science  lessons  were  designed  for  delivery  by  a multimedia  computer.  Two  patterns  of  music 
were  produced  for  each  lesson.  The  first  pattern  caused  the  affective  impact  of  the  musical  selections  to  be  congruent 
with  the  psychological  dynamics  of  the  instructional  strategies.  This  arrangement  was  intended  to  moderate  the  treatment 
to  enhance  achievement.  The  second  pattern  caused  the  affective  impact  of  the  musical  selections  to  be  incongruent  with 
the  psychological  dynamics  of  the  instructional  strategies.  Conversely,  this  arrangement  was  intended  to  moderate  the 
treatment  to  retard  achievement.  Each  lesson  was  also  delivered  with  no  music  or  silence. 

Method 


Subjects 

Subjects  were  48  ninth  grade  students  from  a high  school  in  a middle-class  socioeconomic  metropolitan  area. 
They  were  enrolled  in  a physical  science  class  that  used  very  little  multimedia.  All  of  the  subjects  were  on  a regular 
academic  track. 


Materials 

A single  multimedia  computer  was  used  to  present  the  treatment  lessons.  An  audio  tape  player,  with  detachable 
speakers  was  also  used.  The  computer,  monitor,  and  speakers  were  situated  either  near  the  end  of  a large  conference  room 
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table  or  on  a classroom  counter  for  comfortable  subject  access  (seated)  to  the  presentations.  This  equipment  performed 
flawlessly  throughout  the  experiment.  All  settings  were  void  of  major  distractions. 

Three  lessons  on  physical  science  topics,  designed  especially  for  this  investigation  were  installed  on  the 
computer.  One  lesson  was  about  energy,  another  was  about  machines,  and  the  third  concerned  physical  forces.  They 
concentrated  on  comparing  the  contrasting  types  of  energy,  machines,  and  physical  forces  and  on  measurement  within 
each  of  the  categories. 

The  lessons  were  designed  using  the  instructional  strategies  of  affective  orientation  and  cognitive  dissonance. 

This  was  accomplished  under  the  direct  guidance  of  the  teacher  who  hosted  the  final  phase  of  the  experiment.  She  is  a 
practicing  classroom  teacher,  who  regularly  presents  these  topics  to  her  ninth  grade  class.  Every  effort  was  made  to 
structure  the  lessons  to  be  equivalent  in  form  and  rigor.  The  lessons  were  also  formatively  evaluated  by  several  of  the 
teachers  whose  students  participated  in  the  pilot  studies. 

The  visual  elements  of  the  lesson  presentations  were  developed  using  PowerPoint  (Microsoft,  1992).  Each 
lesson  was  cast  in  its  own,  unique  screen  design  to  distinguish  it  and  to  combat  student  boredom.  The  base  screen  design 
("wizard")  for  each  lesson  was  chosen  for  its  clean,  uncluttered  appearance,  vivid  colors,  and  contrast.  The  lessons  were 
presented  in  the  "slide  show"  mode.  Each  lesson  ran  for  approximately  7.5  minutes.  The  transition  between  screens  was 
a standard,  quick  cut  through  black.  The  parts  of  the  lessons  were  divided  roughly  into  thirds.  The  first  one-third  of  the 
screens  accomplished  affective  orientation.  The  second  third  established  dissonance  by  lifting  up  discrepant  events.  The 
remaining  screens  provided  reconciliation  by  explaining  away  the  dissonance. 

All  screens  contained  two  main  visual  elements,  which  were  text  and  still  graphics.  The  graphics  were  obtained 
from  PowerPoint’s  (Microsoft,  1992)  clip  art  file.  The  words  appeared  on  the  left  side  of  the  screens,  and  the  graphics 
were  on  the  right.  Screens  were  timed  on  the  basis  of  the  number  of  words  they  contained.  Short  messages  of  up  to  four 
words  appeared  for  seconds.  The  time  for  longer 

messages  was  extended  for  one  second  per  additional  two  words.  All  of  the  subjects,  during  the  pilot  studies,  reported 
having  enough  time  to  read  all  screens.  A major  effort  was  made  to  reduce  formal  language. 

Two  sets  of  the  lessons  were  accompanied  alternately  by  music.  The  third  set  ran  in  silence.  The  musical 
. accompaniment  was  simulated  using  the  audio  tape  player  rather  than  internal  to  the  software.  This  was  done  to  achieve 
control  and  precision.  The  use  of  an  audio  tape  player  provided  a sure  control  for  coordination  of  the  musical  passages 
with  the  set  of  screens  it  was  intended  to  support.  It  also  allowed  the  addition  of  a production  value.  Through  the  use  of 
the  volume  control,  the  music  was  faded  in  and  out  at  the  transition  between  sets  of  screens.  The  music  was  played  at 
the  #2  volume  setting  as  was  recommended  during  the  pilot  studies.  A suggestion  was  also  made  during  the  pilot  studies 
that  the  audio  tape  player  not  be  readily  visible  for  fear  of  calling  attention  to  the  music  and  its  manipulation.  This 
recommendation  was  followed,  and  the  player  was  relocated  in  the  final  phase  of  the  experiment  to  be  less  noticeable. 

The  music  accompanying  each  of  the  lessons  was  unique  and  tailored  to  that  lesson.  Each  sound  track  contained 
three  strains  of  classical  music,  based  on  the  advice  of  experts  in  the  field.  Classical  music  was  used  throughout  for 
uniformity  and  because  of  its  unfamiliarity  to  the  subjects.  Classical  music  is  highly  ordered,  and  its  use  added  an  extra 
dimension  of  control  to  this  moderating  variable. 

Table  2 presents  the  works  from  which  the  music  accompanying  each  of  the  lesson  parts  were  drawn.  The  full 
titles  of  the  recordings  are  in  Appendix  A. 


Table  2 

Musical  Selections  by  Lesson 


Lesson 

Congruent 

Incongruent 

Energy 

Beethoven-Egmont 

Gorecki-Sym.#3 

Berg-Chamber  Music 

Bach-Sinfonia 

Elgar-P&C  March  #4 

Albinoni-Adagio 

Forces 

Beethoven-Fidelio 

Holst-Venus 

Berg-Concerto 

Elgar-Nimrod 

Tchaikovsky-March  Slav 

Albinoni-Adagio 
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Machines 


Rossini-Barber  of  Sev. 
Berg-Chamber  Music 
Holst-Jupiter 


Vivaldi-Largo 

Bach-Air 

Marcello-Adagio 


Experimental  Design 

This  study  used  repeated  measures  of  subjects'  immediate  recall  and  understanding  of  lesson  content  together 

with  their  effectiveness  rating  of  the  lessons  supported  by  music.  A Graeco-Latin  Square,  type  GLSFF  3^  design  (Kirk, 
1982)  was  employed  to  obtain  repeated  measures.  This  is  a fractional  factorial  design,  evaluating  varieties  of  two  crossed 
treatments  (visual  lesson  presentations  and  music)  on  individuals  drawn  at  random  from  classes  and  separated  into  three 
groups.  This  design  is  best  represented  by  a cube  with  27  cells.  Only  three  of  the  cells  on  each  of  the  levels  contained 
scores.  Each  cell  containing  scores  occupies  a unique  position  within  the  cube  in  terms  of  rows  and  columns. 

The  treatment  effects  are  both  between  and  within  subjects.  The  design  assumes  that  the  fixed  effects  are  the 
treatments  and  that  the  blocks  are  random  effects.  It  also  assumes  no  interaction  effects. 

Dependent  Measures 

There  were  three  types  of  dependent  measures  in  this  study.  These  measures  included  a posttest  performance  for 
each  lesson,  a subjective  evaluation  (rating)  of  the  interaction  of  music  in  each  lesson,  and  a survey  of  student 
preferences. 

Subjects  were  given  a paper  and  pencil  test  on  lesson  content  immediately  following  each  presentation.  The 
first  part  of  the  test  contained  16  true/false,  multiple  choice,  and  fill-in-the-blank  types  of  questions.  The  order  of  the 
questions  was  arranged  so  that  their  sequence  did  not  parallel  the  lesson  presentation.  The  alpha  coefficients  for  the  tests 
used  in  the  study  were  as  follows:  Energy  test  = .70;  Physical  Forces  test  = .66;  and  Machines  test  = .68. 

Each  test  also  contained  a second  section  where  the  subjects  subjectively  evaluated  the  interaction  of  the  music 
and  the  lesson.  The  subjects  were  asked  to  indicate  whether  they  felt  that  the  music,  when  present,  made  the  content  of 
the  lessons  more  or  less  understandable.  A rating  scale  with  a range  of  1 (less)  to  10  (more)  was  provided. 

Following  the  last  quiz,  the  subjects  were  asked  to  complete  a survey.  It  measured  subjects’  preferences  for 
music  by  type,  music's  presence  in  their  academic  pursuits,  multimedia  science  lessons,  and  favorite  type  of  science 
class. 

Procedure 

Several  pilot  studies  were  run  in  preparation  for  the  formal  experiment.  Pilot  studies  were  conducted  at  five 
sites  during  the  spring  and  early  fall  to  formatively  evaluate  the  components  of  the  experiment.  Two  of  the  pilot  studies 
involved  a full  administration  of  the  experiment.  Three  others  had  special  purposes  involving  equating  the  lessons  and 
achievement  tests  together  with  ensuring  the  reliability  of  the  instruments. 

The  final,  formal  phase  of  the  experiment  required  3 days  to  administer.  Little  discussion  was  needed  with 
subjects  in  order  to  initiate  the  experiment.  They  simply  were  told  that  they  would  view  three  science  lessons  and  would 
take  a quiz  following  each.  To  enhance  their  motivation,  the  subjects  were  told  by  their  teacher  that  their  scores  on  the 
tests  would  constitute  a daily  grade. 

A random  selection  of  the  beginning  cell  yielded  the  machines  lesson  moderated  by  incongruent  music.  With 

this  starting  point  located,  the  GLSFF  3 design  mandated  the  following  order  of  treatment  administrations  to  the 
respective  groups.  The  treatments  for  Group  One  were  the  machines  lesson  with  incongruent  music,  forces  with  no 
music,  and  energy  with  congruent  music.  Group  Two  received  the  lessons  in  the  order  of  forces  with  congruent  music, 
energy  with  incongruent  music,  and  machines  with  no  music.  The  scenario  for  the  third  group  was  the  energy  lesson 
with  no  music,  machines  with  congruent  music,  and  forces  with  incongruent  music.  This  scenario  is  shown  in  Figure  1 . 
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Figure  1.  Administration  of  the  treatments 


Figure  L Viewed  from  top  to  bottom,  the  shaded  cells  contain  scores  for  Groups  1,  2,  and  3.  The  lessons  are  shown  on 
the  right  face  of  the  cube  (E  = energy,  M = machines,  F = physical  force).  The  musical  patterns  are  shown  on  the  left 
face  of  the  cube  (C  = congruent,  I = incongruent,  N = no  music). 

Results 


Posttest  Performance 

The  summary  tables  which  follow  organize  the  data  for  use  in  the  analysis  of  variance  required  by  the  GLSFF 
33  design.  Table  3 presents,  by  group,  the  number  of  test  items  answered  correctly  in  relationship  to  the  lessons. 

Table  3 

Group  Scores  by  Lesson 


Group 

Machines 

Forces 

Energy 

Total 

" 1 

150 

135 

146 

431 

2 

148 

149 

165 

462 

3 

152 

146 

165 

463 

Total 

450 

430 

476 

o i 
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Table  4 presents,  by  musical  pattern,  the  number  of  test  items  answered  correctly  during  each  administration  of 
the  treatments.  It  profiles  the  effect  of  the  musical  patterns  across  the  sequence  of  treatments. 

Table  4 

Group  Scores  by  Musical  Pattern 


Pattern 

Treatment 

Treatment 

Treatment 

Total 

1 

2 

3 

Incongruent 

150 

165 

146 

461 

Congruent 

149 

152 

146 

447 

No  Music 

165 

■ 135 

148 

448 

Total 

464 

452 

440 

Table  5 presents  group  performance  in  the  context  of  all  treatment  components,  where: 
a j = lesson;  j = 0,  machines; 

= 1,  forces; 

= 2,  energy; 

b j = group;  j = 0,  group  1 ; 

= 1,  group  2; 

= 2,  group  3; 

Cj  = music;  j = 0,  incongruent; 

- 1,  no  music; 

= 2,  congruent; 


Table  5 

Total  Number  of  Items  Correct  in  Each  Treatment  Component 


aoboco 

alblc2 

a2b2cl 

150 

135 

146 

alb0cl 

a2blc0 

a0b2c2 

149 

165 

148 

a2b0c2 

“oVi 

alb2c0 

165 

152 

146 
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Tables  6,  7,  and  8 contain,  by  group,  the  totals  of  items  answered  correctly  on  the  achievement  tests,  the  sample 
means,  and  standard  deviations  tor  each  treatment  component.  The  data  are  organized  according  to  major  treatment 
(lesson)  juxtaposed  with  its  moderating  variable  (musical  pattern). 

Table  6 

Group  1:  Items  Correct.  Mean  Scores,  and  Standard  Deviations 


Lesson 

Musical 

Items 

Mean 

SD 

Treatment 

Correct 

Machines 

Incongruent 

150 

9.38 

2.47 

Forces 

None 

135 

8.44 

2.49 

Energy 

Congruent 

146 

9.13 

3.67 

Table  7 

Group  2:  Items  Correct.  Mean  Scores,  and  Standard  Deviations 


Lesson 

Musical 

Items 

Mean 

SD 

Treatment 

Correct 

Forces 

Congruent 

149 

9.31 

2.05 

Energy 

Incongruent 

165 

10.31 

2.71 

Machines 

None 

148 

9.25 

2.59 

Table  8 

GrouD  3:  Items  Correct.  Mean 

Scores,  and  Standard  Deviations 

Lesson 

Musical 

Items 

Mean 

SD 

Treatment 

Correct 

Energy 

None 

165 

10.31 

2.93 

Machines 

Congruent 

152 

9.50 

1.62 

Forces 

Incongruent 

146 

9.13 

2.64 

The  ANOVA  prescribed  (Kirk,  1982)  was  calculated  to  test  for  statistical  significance  of  the  effects  of  the  forms 
of  the  independent  variable,  the  forms  of  the  moderating  variable,  and  the  other  factors  impacting  the  treatments,  i.e.,  the 
groups  and  the  order  of  treatment  administration/repeated  measures.  The  summary  for  the  ANOVA  is  shown  in  Table  9. 
The  factors  A,  B,  and  C represent  respectively  the  lessons,  the  order  factor,  and  the  musical  patterns.  No  factor  produced 
a statistically  significant  effect. 
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Table  9 

Computations  for  Type  GLSFF  3^  Design 


Source 

df 

MS 

F 

Between  blocks 

47 

528.667 

0.027 

Groups 

2 

13.792 

Blocks  w 

45 

514.974 

groups 

Within  blocks 

96 

510.333 

A 

2 

22.166 

0.046 

B 

2 

5.999 

0.013 

C 

2 

2.542 

0.005 

Residual 

0 

0.000 

0.000 

ABC  X blocks 

90 

479.625 

w.  groups 
Total 

143 

1039.000 

Subject  Ratings 

Students  subjectively  evaluated  the  interaction  of  the  music  and  the  lesson  for  both  the  congruent-music  and 
incongruent-music  treatments,  by  indicating  on  a 10  point  scale  whether  they  felt  that  the  music,  when  present,  made  the 
content  of  the  lessons  more  or  less  understandable.  The  t-test  for  non  independent  samples  was  run  on  the  rating  arrays 
for  each  group  and  for  the  total  sample.  There  were  no  statistically  significant  differences  in  the  subjects'  ratings  of  these 
two  musical  patterns.  In  each  case,  the  observed  ratio  was  less  than  the  t-value  required  to  reject  the  null  hypothesis. 

Table  10  compares  the  ratings  by  group  and  for  the  total  sample  of  the  two  musical  patterns.  Means,  standard 
deviations  and  the  t-value  for  these  relationships  are  shown. 

Table  10 

Results  of  the  t-tests 


n 

Congruent 

Incongruent 

t 

X 

SD 

X 

SD 

Group  1 

16 

5.81 

2.23 

6.38 

2.16  ■ 

-1.21 

Group  2 

16 

4.88 

2.16 

4.81 

2.93 

0.12 

Group  3 

16 

5.25 

2.05 

4.44 

2.90 

2.09 

Total 

48 

5.31 

2.14 

5.21 

2.76 

0.38 

Sample 


Table  11  compares  the  relationships  between  subject  achievement  scores  and  the  associated  ratings  for  each 
group  and  for  the  sample  as  a whole.  In  each  case,  the  observed  correlation  coefficient  was  less  than  that  required  to  reject 
the  null  hypothesis. 
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Table  11 

Correlation  of  Ratines  with  Achievement  Scores  on  Different  Musical  Patterns 
Rating  Achievement 

Incongruent  Congruent 


Group  1 

Incongruent  -.17 

Congruent 

Group  2 

Incongruent  .32 

Congruent 

Group  3 

Incongruent  .08 

Congruent 

Total  Sample 

Incongruent  .09 

Congruent 


-.14 


.14 


.00 


-.04 


Student  Preferences 

Individual  student  preferences  were  collected  using  a survey  which  included  questions  about  music,  science 
lessons  and  multimedia.  The  results  indicated  that  the  students  generally  study  in  the  presence  of  music  (77%).  That  was 
consistent  with  the  report  by  81  % of  the  subjects  that  the  musical  patterns  used  in  this  experiment  did  not  distract  them. 
The  failure  of  patterns  of  classical  music  to  exercise  a strong  distraction  effect  is  probably  due  to  the  high  preference 
ratings  given  by  the  subjects  to  rap  (40%),  pop  (17%),  and  light  rock  (14.5%).  These  are  all  forceful  forms.  % of  the 
subjects  of  a liking  for  science  lessons  delivered  in  the  fashion  of  this  experiment.  A large  percentage  (77%)  of  the 
students  indicated  that  they  liked  the  science  lessons  delivered  this  way  and  73%  indicated  that  they  would  take  more 
science  courses  if  they  were  delivered  using  multimedia. 

Discussion 

The  purpose  of  this  study  was  to  assess  the  impact  of  music  as  an  aid  to  learning.  It  attempted  to  enhance 
achievement  by  stimulating  triple  encoding  in  short  term  memory  by  aligning  an  abstract  musical  input,  which 
elaborated  subjectively  on  content  (Burger,  1993),  with  instructional  strategy,  a subjective  lesson  element.  This  research 
tested  music  in  the  role  of  a moderating  variable,  labeled  Music-Instructional  Strategy-Integration  (MISI).  The  rationale 
for  this  moderating  variable  was  to  integrate  abstract  music  with  instructional  strategy,  a subjective  lesson  element,  on 
the  basis  of  psychological  feelings.  Participants  viewed  three  multimedia  science  lessons  that  were  designed  using  the 
instructional  strategies  of  affective  orientation  and  cognitive  dissonance.  Musical  selections  accompanied  the  lessons  and 
were  either  congruent  or  incongruent  with  the  psychological  dynamics  of  the  instructional  strategies.  Lessons  were  also 
delivered  without  music.  The  study  assessed  immediate  recall  and  understanding  of  the  lessons,  students’  subjective 
evaluation  (rating)  of  the  interaction  of  music  in  each  lesson,  and  student  preferences  concerning  music,  science,  and 
multimedia  instruction. 

The  high  MISI  condition  (music  congruent  with  instructional  strategy)  was  expected  to  enhance  student 
achievement  based  on  recent  studies  concerning  the  use  of  music  in  advertising  (Kellaris,  1991;  Kellaris,  Cox  & Cox, 
1993;  Kellaris  & Kent,  1993)  and  in  psychology  (Boltz,  Schulkind,  & Kantra,  1991).  A second  expectation  was  for  the 
low  MISI  condition  to  retard  achievement  compared  to  the  no  music  condition.  We  did  not  find  a statistically  significant 
difference  between  the  three  conditions  - high  MISI,  low  MISI,  or  no  music.  This  finding  is  also  not  in  agreement  with 
the  research  which  regards  music  as  a distraction  (Zillman,  1994). 
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Students'  rated  the  interaction  of  music  in  each  lesson  by  answering  the  question  "Did  the  music  make  the 
material  in  this  lesson  more  or  less  understandable?"  using  a ten-point  scale.  Students  were  expected  to  find  the 
incongruent  music  distracting  and  to  therefore  think  that  the  music  made  the  lesson  less  understandable  in  the  lessons 
with  low  MISI.  It  was  also  expected  that  students  would  think  that  the  music  made  the  lesson  more  understandable  in 
the  lessons  with  high  MISI.  These  expectations  were  not  fulfilled  based  on  the  statistical  analysis.  This  finding  is, 
however,  consistent  with  student  performance  on  the  achievement  tests  and  may  provide  an  explanation  for  our  results. 

This  study  ratified  the  popular  notion  that  ninth  grade  students  prefer  instruction  with  a musical  ingredient.  It 
also  found  that  students  believe  that  music  adds  to  lesson  effectiveness  and  that  the  inclusion  of  music  can  stimulate 
positive  student  affect  toward  computer-supported  science  instruction. 

There  are  several  possible  explanations  for  why  the  insertion  of  music  which  was  congruent  with  the 
psychological  operation  of  the  lessons'  instructional  strategies  did  not  improve  performance.  One  consideration  in  this 
study  was  the  choice  of  classical  music.  We  purposefully  chose  music  that  was  not  familiar  to  these  students.  Familiar 
music  could  possibly  be  more  distracting  and  might  carry  an  affective  association.  The  results  from  the  students' 
subjective  ratings  of  the  music,  however,  showed  that  they  may  not  have  distinguished  between  the  congruent  and 
incongruent  treatments.  The  use  of  classical  music  with  this  particular  group  of  subjects  may  have  acted  similar  to 
background  music. 

The  failure  of  this  technique  to  produce  a significant  effect  could  suggest  that  multiple  encoding  is  an  objective, 
content  specific  phenomenon,  not  under  the  influence  of  subjective  lesson  attributes.  This  study  suggests,  therefore,  the 
need  for  a similar  investigation  of  multiple  encoding  using  only  literal  music.  Literal  music  depicts  content  by 
becoming  one  of  its  objective  parts  as  in  a sound  effect  (Burger,  1993). 

In  designing  this  experiment,  we  purposely  used  a simple  dissemination  strategy  rather  than  embellishing  the 
lessons  with  components  that  are  known  to  improve  performance,  (e.g.  practice,  feedback,  interactivity).  We  did  this  in 
an  attempt  to  avoid  a "no  significant  difference"  finding  due  to  a ceiling  effect.  It  is  possible,  however,  that  the  efficiency 
of  the  instructional  design  employed  and  the  impact  of  the  visual  elements  for  these  lessons  fully  executed  the 
psychology  of  the  instructional  strategies  so  that  no  room  was  left  for  the  music  to  contribute  to  those  psychological 
dynamics.  This  study  suggests  the  need  for  further  experimentation  with  music  as  an  agent  in  dual  encoding  where  the 
music  is  associated  with  only  one  other  message  element  such  as  text.  A related  but  different  explanation  is  the 
possibility  of  a novelty  effect  associated  with  multimedia  delivered  instruction.  These  students  were  not  used  to  this 
delivery  mode  in  their  classroom.  The  uniqueness  of  these  lessons  together  with  the  short  length  of  the  lessons,  may 
have  held  their  attention  more  than  if  it  were  a common  or  lengthier  occurrence. 

This  investigation  clearly  demonstrated  that  adolescents  prefer  to  have  music  present  in  their  academic  activities 
and  may  be  more  interested  in  science  courses  which  utilize  the  use  of  multimedia.  This  finding  together  with  the  mixed 
results  of  research  about  the  best  way  to  integrate  music  into  instruction  strongly  suggests  the  need  for  further  studies. 
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Introduction 


Over  the  last  ten  years,  electronic  mail  (email)  has  provided  a new  vehicle  for  communication.  As  D'Souza  (1992)  stated, 
electronic  mail  has  become  a mainstay  of  organizational  communication.  Its  four  essential  characteristics,  she  continues, 
namely  cost  reduction,  reduced  paper  handling,  faster  communications,  and  improved  communication  effectiveness,  have 
attracted  users  both  from  businesses  and  academia. 

At  Indiana  University  alone,  there  are  over  40,000  active  accounts.  These  include  student,  faculty,  and  staff  accounts.  It  is 
estimated  that  over  80%  of  these  accounts  are  active,  and  99%  of  them  are  used  for  electronic  mail.  Mark  Sheehan  of 
University  Computing  Services  at  Indiana  University  mentioned  that  email  has  been  in  common  use  among  students 
only  since  1988  when  there  were  only  8,000  computer  accounts  issued. 

From  reviewing  the  literature  on  electronic  mail,  we  found  that  researchers  have  not  made  a distinction  between  users 
who  use  electronic  mail  many  times  during  the  day  (high-end  users),  and  casual  users  (low-end  users).  Therefore,  research 
findings  on. how  people  use  and  feel  about  electronic  mail  have  been  reported  without  considering  whether  the  amount  of 
usage  is  a factor  in  the  participants  responses. 

Therefore,  for  this  paper,  the  scope  has  been  narrowed  to  high-end  users.  The  researchers  set  out  to  discover  what  high-end 
users  (students  and  faculty  only)  do  with  email  and  how  they  feel  about  it.  It  is  our  hope  that  in  the  future  similar  studies 
will  be  conducted  with  low-end  users,  in  order  to  be  able  to  compare  and  contrast  studies. 

Literature  Review 

There  is  a respectable  volume  of  research  that  is  devoted  to  the  use  of  email  in  academic  settings.  Poling  (1992) 
encourages  students  and  professors  to  use  electronic  mail  to  their  advantage  for  classroom  purposes.  He  firmly  believes 
that  educators  can  build  a stronger  relationship  with  students  by  simply  spending  a few  minutes  each  day  using  email  to 
communicate  with  their  students.  Moreover,  many  other  researchers,  including  D’Souza  (1992),  Rubcrg  and  Miller 
(1993),  and  Lowry,  Koneman,  Osman-Jouchoux,  and  Wilson  (1994),  suggest  that  students  and  faculty  use  electronic  mail 
for  all  of  the  following:  answering  direct  questions  from  any  student,  counseling,  class  assignments,  general  class 
announcements,  occasional  quizzes,  direct  communication  with  a particular  student,  posting  grades,  helpful  hints  about 
homework  or  upcoming  quizzes  or  tests,  and  excuses  for  missing  class. 

Nevertheless  as  Ruberg  and  Miller  (1993)  stated,  direct  communications  such  as  electronic  mail  may  resolve  some 
communication  problems,  but  it  may  also  introduce  some  others.  For  example,  the  physical  aspects/personalization  of 
communication  such  as  face-to-face  interactions  and  hard  copy  formats  are  lost,  and  some  users  of  email  complain  about 
information  overload  preventing  them  from  managing  all  the  email  they  receive. 

Design  of  Study 

Sample 

Our  sample  consisted  of  12  people  in  the  Instructional  Systems  Technology  (1ST)  department  at  Indiana  University.  We 
chose  1ST  faculty  and  students  because  we  recognized  that  they  represent  a unique  population.  First,  the  students  in  1ST 
are  required  to  pass  a computer  proficiency  which  includes  email.  Second,  the  faculty  and  students  deperu  on  email  as  the 
sole  medium  for  posting  job  announcements,  class  announcements  and  community  events.  Third,  IS7  as  somewhat 
unique  because  of  the  high-end  use  of  computers.  Each  of  our  subjects  checked  their  email  daily.  Fourth,  members  of  the 
1ST  community  represent  a large  number  of  students  from  different  backgrounds.  Not  only  are  there  students  from  around 
the  world,  but  also  from  many  disciplines.  Finally,  the  fact  that  1ST  is  a graduate-only  program  makes  it  unique. 

We  divided  the  1ST  department  into  three  major  groups,  the  faculty  (F1-F4),  non-first  year  students  (S1-S4),  and  first  year 
students  (S5-S8)  and  chose  four  subjects  to  represent  each  group,  F1-F4  and  S1-S8.  Within  this  sample,  we  tried  to 
mirror  the  diversity  of  1ST.  Overall,  we  had  five  females  and  seven  males.  Nine  of  our  subjects  were  American,  the  other 
three  were  international  students. 
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The  choice  to  split  up  the  students  into  non-first  year  and  first  year  was  based  on  the  assumption  that  length  of 
experience  with  email  may  influence  opinion.  Many  of  the  first  year  students  may  be  newer  to  the  technology  than  the 
established  students.  The  faculty  were  chosen  because  their  point  of  view  is  relevant  to  the  way  1ST  uses  email,  yet 
potentially  differs  from  the  ideas  of  the  students. 

Procedure 

The  four  researchers  each  interviewed  one  member  from  each  of  our  major  subject  groups.  Each  interview  was  audio 
taped.  We  asked  our  respondents  how  often  they  send  and  receive  email,  how  they  use  it,  and  how  they  feel  about  email. 
The  researchers  then  transcribed  the  interviews  and  returned  them  to  each  respondent  for  a member  check. 

Once  the  member  checks  were  completed,  the  researchers  broke  the  transcriptions  into  idea  units  and  placed  them  on  note 
cards.  The  question  that  was  being  answered  in  each  case  was  also  put  on  each  notecard  to  ensure  that  the  information  was 
not  being  taken  out  of  context.  Finally,  the  notecards  were  grouped  and  sorted  into  categories.  From  there,  the  analysis 
took  place. 

Findings 

Frequency  of  Use 

All  of  the  subjects  reported  checking  email  at  least  once  a day.  Most  of  them  report  checking  it  two  to  ten  times  a day. 
Two  of  the  faculty  members  keep  email  open  all  the  time.  This  means  that  they  are,  in  essence,  checking  their  email 

constantly. 

The  subjects  report  receiving  generally  five  to  forty  email  messages  each  day.  However,  they  only  claim  to  send  two  to 
ten  messages  each  day.  Several  of  the  respondents  mentioned  that  a lot  of  the  email  they  receive  is  from  listservs  they  are 
subscribed  to.  This  helps  explain  the  discrepancy  between  the  number  of  messages  sent  and  the  number  of  messages 
received. 

Purpose  of  Use 

When  the  data  was  analyzed,  three  major  categories  of  usage  emerged.  Our  subjects  generally  use  email  for  personal 
communication,  class  related  communication  and  collaboration  with  colleagues. 

Personal 

In  the  area  of  personal  use,  email  proves  to  be  a cheaper  means  of  communication  than  long  distance  phone  calls 
and  is  more  likely  to  be  written  and  sent  than  a letter.  Several  of  the  subjects  acknowledge  that  it  is  email  that 
allows  them  to  keep  in  touch  with  people  who  would  have  fallen  out  of  their  lives.  As  one  person  pointed  out, 

“I  never  would  have  had  any  track  of  them  (friends  from  high  school)  at  all.”  (FI)  Another  person  also  says  that 
email  helps  her  keep  in  touch  with  “a  person  you  like,  but  they  are  not  intimate  enough  to  spend  long  distance 
money  on.”  (S3) 

In  a related  use,  one  international  student  mentioned  that  while  she  was  adjusting  to  life  in  Indiana,  email 
become  her  connection  with  her  home: 

When  I first  came  here,  I was  using  it  a lot  to  send  messages  to  my  friends  in  Taiwan.  In  the 
beginning,  when  I came  here,  1 was  feeling  very  lonely  so  I was  using  it  to  talk  to  my  friends  a lot, 
but  now  I feel  comfortable  here,  so  1 don’t  use  it  as  much  as  before.  (S7) 

In  personal  communications,  besides  the  economic  benefit,  there  are  two  other  benefits  to  email  that  were  stated 
by  our  subjects.  First,  the  ability  to  send  it  to  many  people  at  once.  This  allows  for  easier  handling  of  logistics 
such  as  planning  group  meetings.  Next,  the  subjects  mentioned  the  ability  to  use  listservs.  One  faculty  member 
stated: 


From  what  I've  seen,  most  listservs  tends  to  have  one  or  two  people  on  them  that  know  what  they’re 
talking  about,  and  hundreds  and  hundreds  of  people  who  don't  - who  are  either  asking  questions,  or 
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they  are  getting  on  there  and  they  are  sort  of  yammering  the  same  platitudes,  truisms  that  they  read  in 
Newsweek  last  week.  (FI) 

But,  she  continued  that  she  benefits  from  this  kind  of  communication  with  people  who  have  similar  interests. 
Class  Use 

Both  our  faculty  and  student  respondents  applaud  email  for  class  uses.  The  students  report  that  it  is  a wonderful 
way  to  organize  group  and  project  activities  and  meetings.  They  also  like  using  email  to  communicate  with 
professors.  One  student  stated 

I would  be  less  inclined  to  go  and  talk  to  a professor.  I would  be  more  likely,  I mean  definitely  more 
likely,  to  send  them  an  email  than  to  call  them.  I would  probably  never  call  a professor,  but  I would 
probably  go  into  their  office  before  I would  call  them.  But,  sending  an  email  is  no  big  deal.  (S6) 

The  students  also  use  email  to  continue  class  discussions,  thereby  extending  learning  beyond  the  walls  of  the 
classroom.  Finally,  a few  students  mentioned  that  they  sometime  submit  assignments  via  email. 

Like  the  students,  the  professors  like  email's  effect  on  class  communications.  They  mentioned  the  convenience 
of  sending  out  class  information  over  email  instead  of  trying  to  find  a different  way  to  communicate  with  the 
students.  They  are  also  able  to  answer  the  student's  questions  both  quickly  and  at  their  own  convenience.  As  one 
professor  mentioned: 

Where  formerly  a student  may  have  made  an  appointment  or  I would  have  felt  compelled  to  have  office 
hours  where  I sit  in  the  office  and  stare  at  the  door  and  make  sure  I don't  do  anything  else  because  that's 
the  only  time  students  can  get  at  me.  Now  I know  students  who  have  questions  can  ask  them  24  hours 
a day  and  I can  answer  them,  sometimes  in  the  middle  of  the  night  or  ...  on  weekends. (F3) 

The  professors,  like  the  students,  also  mentioned  the  expansion  of  learning  time  past  the  one  to  three  hours 
reserved  for  classes. 

Collaboration 

Finally,  both  students  and  faculty  cited  collaboration  as  a major  use  of  email.  They  use  email  to  consult  with  other 
professionals  and  students,  make  initial  contacts  with  people  who  they  would  like  to  collaborate  with,  organize  projects, 
and  maintain  co;>  xt  with  colleagues.  Perhaps  the  best  summary  of  the  strength  of  email  for  these  kinds  of  activities 
comes  from  one  professor  who  said,  "(email  is)  especially  nice,  for  instance,  in  the  case  of  [a  colleague]  in  Venezuela. 
There's  hardly  any  other  way  to  talk  to  Venezuela.  The  phone  system  doesn't  work  well  - the  mail  doesn't  work  at  all." 
(F3)  He  goes  on  to  state  that  because  of  email,  this  colleague  is  able  to  serve  on  an  1ST  student’s  dissertation  committee. 

Feelings  About  Use 

Once  the  data  was  interpreted  and  separated,  there  appeared  to  be  five  distinct  categories  that  people's  feelings  about  email 
use  fit  into.  On  the  positive  side,  they  claimed  they  like  email  because  it  is  unobtrusive  and  convenient  and  efficient.  The 
negative  feelings  included  email  being  overwhelming,  it  is  problematic  to  communicate  feelings  via  email  and  email  is 
not  useful  for  complex  issues. 

All  of  our  subject  mentioned  the  convenience  of  using  email.  They  really  enjoy  the  freedom  of  being  able  to  ask 
questions  whenever  they  want,  and  being  able  to  answer  questions  at  their  leisure.  Because  email  is  unobtrusive,  no  one 
hesitates  to  end  an  email  if  they  need  to  communicate  with  a colleague  or  student/professor.  As  one  student  (S3)  stated,  "I 
think  that  I like  email  because  it's  unobtrusive  - because  when  somebody's  checking  their  email,  they  are  choosing  to 
check  it  and  they  can  determine  what  they  want  to  do  with  the  message."  This  same  student  reemphasized  her  point  by 
stating  that  "it's  (email)  is  different  than  calling  someone  on  the  phone,  because  a person  can't  control  when  they  are 
going  to  get  a phone  call,  but  you  can  control  when  you  check  your  email,  and  how  you  want  to  deal  with  it."  This 
feature  of  email  makes  it  a very  convenient  communication  device. 
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The  efficiency  of  email  makes  it  highly  appreciated.  Email  allows  the  sender  more  time  to  formulate  questions  and  be 
sure  the  question  being  asked  is  the  intended  question.  It  also  provides  a means  for  people  to  get  answers  faster  than  they 
would  if  they  had  to  reach  a person  by  telephone  or  personally.  As  one  professor  pointed  out,  “I  think  that’s  a neglected 
attribute  of  email  - that  it  can  answer  problems  more  quickly  than  you  might  otherwise  do  face-to-face.”  (F3) 

The  major  complaint  about  email  that  our  subjects  offered  is  that  they  receive  too  much  of  it.  They  mainly  blame 
listservs  for  the  bulk  of  their  email.  As  one  faculty  member  summed  up  the  situation: 

The  whole  backside  of  email  is  managing  the  incoming  and  outgoing  messages  . . . that's  a whole  other  world 
of  use  and  a troublesome  one  . . . it's  a huge  clerical  job  that  lands  on  your  desk  all  of  a sudden,  and  I don  t 
really  know  to  what  extent  we  fall  backwards  into  our  picture  of  how  people  spend  their  work  time,  but  it's 
there.  (FI) 

Communication  of  feelings  is  another  area  in  which  email  falls  short.  As  a faculty  member  pointed  out,  “We’ve  all  had 
bad  experiences  or  misunderstandings,  hurt  feelings  over  something  that  someone  typed  and  it  didn’t  come  across  right  to 
the  other  person.”(Fl  ) Several  of  the  students  also  found  this  to  be  a problem. 

Finally,  our  subjects  felt  that  email  is  not  appropriate  for  complex  issues.  One  student  summarized  this  point  of  view  by 
stating,  “There  are  certain  sorts  of  chit-chatty  things  you  just  can’t  type  at  all.”  (S3) 


Conclusions 

There  are  several  conclusions  that  can  be  drawn  from  this  research.  First,  there  is  a problem  with  a deluge  of  email. 
Unlike  D’Souza,  we  found  that  email  is  an  additional  form  of  communication.  It  has  not  replaced  any  other  forms, 
merely  provided’  a new  form.  In  fact,  as  Ruber  and  Miller  pointed  out,  email  has  caused  some  new  communications 
problems.  For  instance,  several  of  our  subjects  mentioned  that  it  has  caused  a redundancy  in  communication.  For 
instance,  they  receive  a phone  call  telling  them  to  check  their  email  which  contains  a letter  which  has  also  been  faxed  to 
them.  This  is  not  an  effective  use  of  the  technology  and  does  not  improve  the  effectiveness  of  their  communication. 

Next,  there  is  definitely  a cost  and  time  efficiency  benefit  to  using  email.  Our  data  matched  D’ Souza’s  findings  - email 
provides  faster  communications.  It  also  allows  the  sender  and  receiver  the  freedom  to  deal  with  their  questions  at  their 
own  convenience  instead  of  having  to  coordinate  with  other  people  to  meet  them  face-to-face. 

Finally,  we  noted  that  our  subjects  report  that  their  feelings  about  using  email  have  changed  over  time.  Each  ot  them 
reported  being  very  excited  about  using  it  at  first.  They  sent  messages  to  everyone  they  knew  who  had  email  because  it 
was  fun  and  exciting.  After  even  a few  months  of  using  email,  they  view  it  merely  as  a communication  medium.  They 
feel  it  is  something  that  needs  to  be  dealt  with  daily.  While  they  are  not  unhappy  about  having  email,  our  subjects  are 
not  excited  by  it  anymore  either. 

Future  Research 


This  report  focuses  only  on  a very  narrow  group  of  email  users.  In  the  future,  research  should  be  done  to  find  out  how 
other  groups  of  people  feel  about  email.  It  would  be  interesting  to  compare  high-end  users  to  low-end  users  not  only  int 
he  ways  they  use  the  medium,  but  also  how  they  feel  about  the  medium.  Another  area  of  research  would  be  to  compare 
use  of  email  to  that  of  other  communication  mediums  - especially  looking  at  the  frequency  of  use  and  types  of  use  for 
each. 


Bibliography 

D’  Souza,  P.  M.  (1992).  Electronic  Mail  in  Academic  Settings:  A Multipurpose  Communications  Tool.  Educational 
Technology,  32(3),  22-25. 

Lowry,  M„  Koneman,  P.,  Osman-Jouchoux,  R.,  & Wilson,  B.  (1994).  Electronic  Discussion  Groups:  Using  Email  as 
an  Instructional  Strategy.  Tech  Trends,  39(2),  22-24. 

Poling,  D.  (1994).  Email  as  an  Effective  Teaching  Supplement.  Educational  Technology.  34(5),  53-55. 


2 H l 


268 


Ruberg,  L.,  Miller,  M.  (1993).  Moving  from  U.S.  Mail  to  Email.  In  Art,  Science,  and  Visual  Literacy:  Selected 
Readings  from  the  Annual  Conference  of  the  International  Visual  Literacy  Association. 

Sheehan,  M.  Information  cited  in  University  Computing  Services  Documents. 


2 


u 


269 


Title: 


Cognitive  Strategies  and  the  Use  of  a Hypermedia  Information  System: 

An  Exploratory  Study 


Authors: 

Janette  R.  Hill 
University  of  Georgia 

and 

Michael  J.  Hannafin 
University  of  Georgia 


Abstract 

The  purpose  of  this  study  was  to  identify  the  strategies  used  by  learners  in  an  open-ended  hypermedia 
information  system.  Four  participants  were  drawn  from  an  introductory  technology  course  incorporating  a unit  on 
telecommunications.  Participants  completed  a survey  measuring  reported  knowledge  in  three  domains  (metacognitive, 
system,  and  subject)  as  well  as  self-efficacy  toward  technology.  Participants  searched  for  information  using  Netscape ®, 
thinking  aloud  as  they  searched.  Data  were  collected;  analysis  occured  in  several  phases:  scripting  the  search,  reading 
through  the  data,  segmenting  according  to  research  question,  and  encoding.  Three  major  findings  related  to  hypermedia 
information  systems  resulted  from  this  study:  a variety  of  strategies  are  used  by  learners;  reported  knowledge  does  have 
an  affect  on  the  strategies  used;  and  perceptions  of  disorientation  and  level  of  perceived  self-efficacy  have  an  affect  on  the 
strategies  used.  Implications  related  to  emerging  information  technologies,  open-ended  learning  environments,  and  the 
ways  learners  think  are  considered. 


INTRODUCTION 

Interest  in  creating  productive  and  stimulating  learning  environments  is  an  enduring  theme  in  the  educational 
arena.  Recent  attention  has  focused  on  open-ended  learning  environments  (OELEs).  In  OELEs,  experience  and  context  are 
critical  for  cultivating  cognitive  processes  that  support  understanding.  These  environments  are  learner-centered,  with  the 
instructor  adopting  the  role  of  facilitator  as  opposed  to  presenter.  The  learner  is  encouraged  to  explore  and  experiment; 
problem  solving,  critical  thinking,  and  perspective-building  are  inherent  processes  in  these  environments.  While  learning 
outcomes  remain  important  to  knowing,  processes  are  the  focal  point  of  growth  in  understanding  (Hannafin,  Hall,  Land, 
& Hill,  1994;  Papert,  1993). 

The  emphasis  on  cultivating  processes  and  strategies  increases  the  requirement  for  learner-supplied  structure. 
While  the  instructor  or  system  can  assist  by  guiding  and  facilitating,  the  learner  determines  which  tools  and  resources  to 
access  and  manipulate;  that  is,  how  to  navigate  to  promote  learning  (Bransford  et  al.,  1986;  Spiro,  Feltovich,  Jacobson, 
& Coulson,  1991).  The  individual  determines  what  s/he  wants  or  needs  to  know,  for  accessing  the  information  in  the 
system,  and  for  deciding  whether  or  not  the  system  contains  the  needed  information  (Perkins,  1993;  Roth  & 
Roychoudhury,  1993). 

Numerous  information  systems  exhibit  some  characteristics  of  OELEs.  These  systems  blur  traditional 
distinctions  between  educational  systems  and  information  retrieval  systems.  They  have  become  increasingly  transparent 
and  user-centered.  Emerging  information  systems  provide  the  user  with  more  control  through  “point  and  click” 
technology,  and  expand  the  retrievable  resources  to  include  multiple  media. 

Emerging  information  systems  do  more  than  archive  and  retrieve  data;  they  support  the  individual's  efforts  to 
learn,  think,  and  understand  (Hannafin,  1992).  These  systems  are  no  longer  the  exclusive  domain  of  information 
scientists.  They  are  readily  accessible  and  usable  by  the  consumer,  creating  both  extraordinary  opportunities  for  student- 
centered  learning,  as  well  as  formidable  challenges  for  those  who  design  and  develop  such  systems. 

The  challenges  associated  with  creating  electronic  information  systems  that  function  as  OELEs  are  considerable. 
Some  systems  are  simplistic  and  focused;  others  are  complicated  and  virtually  limitless  in  breadth  and  depth.  Designing 
and  developing  electronic  information  systems  that  empower  the  user,  are  intuitive  and  self-evident,  and  inclusive  in 
orientation  is  a formidable  challenge  (Norman,  1988).  It  has  also  proven  difficult  to  assist  and  support  the  learner  in 
their  uniquely  individual  goals  as  they  use  such  systems.  Learners  accustomed  to  interacting  in  environments  that  are 
prescriptive  and  directed  are  often  inefficient  and  ineffective  in  their  use  of  and  interactions  within  open-ended,  learner- 
centered  environments.  They  may  lack  the  orientation,  mental  model,  or  strategies  necessary  for  the  environments  (Hill, 
Lebow,  Driscoll,  & Rowley,  1994). 

Another  concern  is  that  emerging  information  systems  have  evolved  despite  the  absence  of  a strong  theoretical 
or  research  foundation.  We  lack  a solid  understanding  as  to  how  or  why  the  systems  are  being  used.  In  addition,  we  lack 
a clear  conceptualization  of  the  capabilities  and/or  features  that  are  needed  or  desired. 

The  purpose  of  this  study  was  to  identify  the  strategies  used  by  learners  seeking  information  in  an  open-ended 
hypermedia  information  system.  The  following  questions  provided  a framework  for  this  study. 

Primary  Question 

What  strategies  are  employed  in  open-ended  hypermedia  information  systems? 

Secondary  Questions 

1 . Does  metacognitive  knowledge  affect  the  strategies  employed? 

2.  Are  perceptions  of  disorientation  influenced  by  the  strategies  employed? 

3.  Does  perceived  self-efficacy  affect  the  strategies  employed? 

4.  Does  system  knowledge  affect  the  strategies  employed? 
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5.  Does  subject  knowledge  affect  the  strategies  employed? 


METHODOLOGY 

Selection  and  Description  of  Participants 

The  population  of  this  study  were  current  and  prospective  educators  enrolled  in  a university-level  technology  for 
educators  course.  This  included  Pre-K  through  12  public  school  teachers  as  well  as  college/university  instructors.  This 
sample  and  population  was  chosen  for  several  reasons:  a lack  of  research  examining  adult  use  of  hypermedia  information 
systems,  willingness  to  participate  in  an  intensive  study,  and  diversity  of  the  participants.  The  university  serves  both 
undergraduate  and  graduate  students  from  the  state,  region,  and  nation,  as  well  as  international  students.  The  students 
reflected  a broad  range  of  experiences  and  backgrounds,  assuring  diversity  in  the  population. 

The  Course:  Technology  for  Teachers 

'Technology  for  Teachers”  attempted  to  create  the  conditions  under  which  people  typically  understand  things: 
first  they  develop  a need  to  learn,  then  subsequently  satisfy  the  need  through  progressive  experiences.  The  course 
operated  under  several  important  premises.  First,  within  educational  settings,  technologies  are  not  so  much  a curriculum 
as  sets  of  tools  which  can  be  put  to  productive  uses.  The  course  worked  to  help  learners  to  establish  what  those 
productive  uses  might  be. 

Another  premise  was  that  productive  use  is  influenced  by  context,  audience,  and  activity.  Problems  presented 
during  the  course  were  anchored  in  everyday  classroom  teaching  and  learning  contexts  for  which  various  technologies  can 
be  employed  for,  and  with,  students,  teachers,  and  communities.  The  goal  was  not  to  become  an  expert  in  technology, 
but  to  become  more  facile  in  teaching  through  technology. 

The  final  premise  was  that  it  was  critical  to  understand  the  processes  associated  with  various  technology 
applications.  Technology  continues  to  advance  at  an  exponential  rate.  Existing  technologies  will  inevitably  be  replaced 
by  systems  that  are  faster,  and  more  powerful.  Therefore,  problem  solving  and  cognitive  strategies  for  the  use  of 
technology  were  highlighted,  rather  than  mastery  of  specific  software  and  hardware. 

Technology  for  Teachers  was  chosen  for  several  reasons: 

• Since  the  course  was  an  elective,  most  students,  both  undergraduates  and  graduates,  entered  with  a high 
level  of  interest  and  motivation. 

• Given  the  recent  movement  to  integrate  technology  into  the  classroom,  the  learners  entered  with  a high 
degree  of  interest  in  learning  various  technologies,  as  well  as  diverse  techniques  for  applying  technology  in 
their  individual  settings. 

• The  course  itself  reflected  the  assumptions  and  principles  associated  with  open-ended  learning 
environments  (OELEs),  including  the  development  of  cognitive  strategies  and  higher-order  thinking  skills 
(see,  for  example,  Hannafin,  Hall,  Land,  & Hill,  1994).  An  inherent  aspect  of  OELEs  is  that  the  learners 
establish  goals  they  want  to  accomplish,  as  well  as  determine  the  steps  needed  to  attain  the  goals.  The 
course  naturally  supported  the  participants  in  the  use  of  hypermedia  information  systems. 

• Telecommunication  technologies  were  integral  to  the  course  activities.  A major  focus  of  the  unit  was  the 
Internet.  Most  learners  were  aware  of  the  Internet,  even  if  they  did  not  understand  what  it  was  or  how  it 
could  be  used  in  an  educational  setting.  The  majority  had  a high  degree  of  curiosity  and  motivation  to  learn 
about  the  Internet , what  it  can  do,  and  the  ways  in  which  it  can  be  used. 

The  Information  System:  Netscape® 

The  system  selected  for  the  study  was  Netscape®,  a hypermedia  information  system  built  upon  World  Wide 
Web(W3)  technologies.  The  system  displays  the  following  characteristics: 

• open-ended  in  nature,  picviding  access  to  a variety  of  information  and  media 

• requires  generative  activities,  placing  the  learner  in  the  driver’s  seat 

• necessitates  an  orientation  toward  discovery,  where  the  learner  is  free  to  explore  vast  resouces 

• control  of  the  environment  centers  around  the  user,  both  in  terms  of  the  information  requested  and  the 
steps  taken  to  retrieve  the  information 

The  World  Wide  Web  is  an  interconnection  of  computer  sites  on  the  Internet.  The  main  difference  between 
World  Wide  Web  technologies  and  traditional  Internet  programs  is  that  W3  allows  for  the  transmission  of  pictures,  sound 
and  motion  along  with  traditional  textual  information.  It  also  differs  from  traditional  Internet  programs  in  that  it  creates 


a point  and  click  interaction  environment,  where  a mouse  or  track  ball  can  be  used  to  retrieve  information.  This  creates 
an  environment  where  hypertext  and/or  hypermedia  capabilities  are  readily  accessible. 

Netscape®  is  a front-end  browser  used  to  interface  with  the  Web.  Netscape®  and  W3  operate  in  a client/server 
environment:  the  program  that  resides  on  the  local  computer  is  the  client,  while  the  server  is  the  remote  location 
accessed.  Netscape®  is  the  browser  that  displays  the  information  retrieved  from  the  Web. 

Method  and  Design 

An  embedded  case  study  approach  was  employed,  involving  the  use  of  multiple  cases,  or  embedded  units,  within 
a larger  context.  The  unit  of  study  in  this  case  was  the  individual  user  of  the  system  from  whom  several  data  sources 
were  gathered.  A unique  strength  of  the  case  study  strategy  is  the  ability  to  deal  with  multiple  sources  of  evidence. 
Various  sources  of  evidence  from  the  embedded  units  allowed  for  triangulation  of  the  data,  as  well  as  serving  to  address 
concerns  with  construct  validity  (Yin,  1994). 

The  methods  combined  exploratory,  descriptive,  and  analytical  approaches.  The  study  was  exploratory  in  that 
several  of  the  methods  have  not  been  previously  used  to  examine  the  use  of  strategies  in  hypermedia  information 
systems.  The  research  was  descriptive  in  that  the  goal  was  to  describe  the  strategies  employed  from  the  evidence 
gathered,  including  the  perspectives  of  the  participants  and  the  researcher.  Information  gathered  during  the  research  was 
manipulated  iteratively  through  analytical  induction.  Initial  definitions  and  explanations  were  developed  at  the  beginning 
of  the  study.  These  definitions  and  explanations  were  then  contrasted  with  the  data  collected  and  refined  based  on  analysis 
of  the  evidence  (Bogdan  & Biklen,  1992). 


Measures  and  Instrumentation 

A combination  of  rationalistic  and  naturalistic  techniques  was  used.  Rationalistic  (quantitative)  techniques, 
including  surveys  and  questionnaires,  were  used  to  generate  individual  difference  measures  for  each  case,  and  audit  trails 
were  used  to  track  the  participants*  movements  as  they  searched  using  Netscape®.  Naturalistic  (qualitative)  techniques, 
included  think-aloud  protocols  and  interviews.  These  methods  were  used  to  monitor  the  participants’  search,  as  well  as  to 
capture  their  thoughts  both  during  and  after  the  search  process. 

A fundamental  theme  in  naturalistic  studies  is  the  element  of  continuous  improvement.  It  is  a widely  held 
belief  in  qualitative  research  that  research  is  a process  that  occurs  on  a continuum,  one  moving  ever  closer  to  a contextual 
reality.  In  keeping  with  this  theme,  and  to  assist  the  researcher  in  creating  a more  complete  picture  of  contextual  reality, 
the  techniques  described  below  were  first  used  in  a developmental  study. 

Figure  1 illustrates  the  progression  of  the  activities  throughout  the  study.  Several  activities  took  place  during 
the  study,  which  can  be  divided  into  three  main  components:  pre-search  activities,  activities  during  the  search,  and  post- 
search activities.  Five  techniques  were  used  to  gather  data:  pre-search  surveys,  think-aloud  protocols,  audit  trails,  post- 
search questionnaires,  and  stimulated  post-search  interviews. 

Pre-search  survey.  The  pre-search  survey  measured  the  learners  intial  perceptions  of  the  three  domain  areas 
(metacognitive,  subject,  system)  as  well  as  their  perceived  self-efficacy. 

Think-aloud  protocols.  Think-aloud  protocols  were  recorded  on  both  audio  and  video  tape  as  the  learners  worked 
in  the  hypermedia  information  system.  Learners  were  encouraged  to  verbalize  whatever  came  to  mind  as  they  worked  in 
the  system. 

Audit  trails.  Video  was  used  to  record  the  learner’s  path  in  the  World  Wide  Web.  The  recordings  were  used  to 
generate  audit  trails  of  the  search. 

Post-search  questionnaires.  Like  the  pre-search  survey,  the  post-search  questionnaire  measured  the  learners 
perceptions  of  the  three  domain  areas  (metacognitive,  subject,  system)  as  well  as  their  perceived  self-efficacy.  In  addition, 
the  questionnaire  examined  the  learner’s  perceived  strategies  as  they  worked  in  the  system. 

Stimulated  post-search  interviews.  Following  the  transcription  of  protocols  and  the  creation  of  audit  trails, 
learners  were  interviewed  to  gamer  their  reflections,  reactions,  and  comments  on  the  search  task. 


See  Appendix  B of  the  J996AECT  Proceedings 

The  techniques  enabled  the  researcher  to  establish  a rich  orientation  to  the  individual’s  search  process,  procedures 
used,  and  strategies  engaged.  Those  techniques  were  used  to  determine  the  five  self-reported  knowledge  areas: 
disorientation,  perceived  self-efficacy,  metacognitive  knowledge,  system  knowledge,  and  subject  knowledge. 


Disorientation.  Disorientation  refers  to  a loss  of  one’s  bearings.  Disorientation  is  often  associated  with  a lack 
of  understanding,  or  an  inability  to  recognize  how  to  use  the  system  to  accomplish  one’s  goals. 

Perceived  self-efficacv.  Self-efficacy  refers  to  a person’s  perceptions  of  ability  to  succeed  on  a given  task. 
Perceived  self-efficacy  refers  to  a person’s  judgments  of  her  or  his  capabilities  to  organize  and  execute  action  required  to 
attain  designated  performance  (Bandura  in  Olivier  & Shapiro,  1993).  Perceived  self-efficacy  transcends  knowing  what  to 
do  or  awareness  of  one’s  skills;  it  is  a judgment  of  how  well  one  can  perform  using  particular  knowledge  or  skill. 

Metacognitive  knowledge.  Metacognition  refers  to  awareness  about  cognition.  Metacognitive  knowledge 
enables  an  individual  to  reflect,  evaluate  and  direct  cognitive  activities  more  effectively  (Perkins,  Simmons,  & Tishman, 
1990).  In  this  study,  metacognitive  knowledge  referred  to  the  participant’s  self-reported  awareness  of  their  cognitive 
processes.  Users  with  extensive  metacognitive  knowledge  were  presumed  able  to  monitor  their  learning  needs,  leading  to 
purposeful  system  use. 

System  knowledge.  System  knowledge  refers  to  the  participant’s  reported  knowledge  about  electronic 
information  systems  in  general,  as  well  as  Netscape®,  in  particular.  System  knowledge  was  felt  to  influence  the  ability 
to  search  successfully  in  an  information  system.  Users  with  extensive  system  knowledge,  such  as  reference  librarians, 
are  often  able  to  use  it  to  maneuver  through  information  systems,  despite  a lack  of  significant  prior  subject  or  domain 
knowledge. 

Subject  knowledge.  Subject  knowledge  refers  to  estimates  of  existing  knowledge  and  experience  related  to  the 
subject  for  which  searches  were  conducted.  Subject  knowledge  was  presumed  to  influence  the  strategies  participants 
employed  as  they  searched  using  Netscape®. 

METHODS,  ANALYSIS  AND  PROCEDURES 

Data  analysis  was  conducted  at  several  stages.  The  developmental  study  provided  an  opportunity  to  test  the 
clarity  of  the  questions  and  terminology,  the  techniques  employed  with  the  think-aloud  protocol,  and  the  accuracy  of  the 
technology  employed  to  gather  data.  It  also  provided  insight  into  the  strategies  being  employed  as  people  work  in  these 
systems. 

Primary  Research  Phase 

Preliminary  Organization  and  Analysis 

Most  of  the  collection,  organization,  and  analysis  of  data  occurred  concurrently.  This  helped  to  indicate  gaps  in 
data  as  they  were  gathered  and  organized,  allowing  adaptations  in  the  process  and  indicating  the  need  for  additional 
information  (see,  for  example,  Glaser  & Strauss,  1967;  Hert,  1992).  Formal  analysis,  comprising  several  reviews  of  the 
data,  took  place  toward  the  end  of  data  collection  (Bogdan  & Biklen,  1992).  Overall  organization  of  the  data  gathered  in 
relationship  to  the  research  questions  is  illustrated  in  Table  1 . 


Table  1.  Research  question,  data  sources,  and  methods 


Research  Question 

Data  Gathered 

How  Obtained 

When 
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1.  Does  metacognitive 
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strategies  employed? 
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Think-Aloud 
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Search 
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Video  tape 

Search 
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Written  responses 
to  several  questions 

Post-search 

Post- 

Participant 

Question. 

search 

Verbal  responses  to 

Stim.  Post-Search 

Post- 

Participant 

several  questions 

Inter. 

search 
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Researcher 
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Table  1.  (continued) 


Research  Question 

Data  Gathered  I 

How  Obtained 

When  1 

Who 

2.  Are  perceptions  of 

Protocol 

Think-Aloud 

Search 

Participant 

disorientation  influenced 

Protocol 

by  the  strategies 
employed? 

Audit  Trail 

Videotape 

Search 

Participant 

Written  responses 

Post- 

Participant 

to  several  questions 

Post-Search 

Question. 

search 

Participant 

Verbal  responses  to 

Post- 

and 

several  questions 

Stim.  Post-Search 
Inter. 

search 

Researcher 

3.  Does  perceived  self- 

Written  responses 

Pre-Search  Survey 

Pre-search 

Participant 

efficacy  affect  the 
strategies  employed? 

to  several  questions 

Post-search  Ques. 

Post- 
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to  several  questions 

Stim.  Post-Search 

search 
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Inter. 
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and 
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several  questions 
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4.  Does  system 
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Video  tape 

5.  Does  system 
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6.  Does  subject 
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Table  1.  (continued) 


Research  Questions 

Data  Gathered 

How  Obtained 

When 

Who 

Overall  Question: 

Written  responses 

Survey 

Pre-search 

Participant 

What  strategies  are 
employed  in  open-ended 

to  several  questions 

hypermedia  information 

Protocol 

Think-Aloud 

Search 

Participant 

systems? 

Audit  Trail 

Protocol 

Search 

Participant 

Videotape 

Written  responses 

Post- 

Participant 

to  several  questions 

Post-Search 

Questionnaire 

search 

Verbal  responses  to 

Participant 

several  questions 

Post- 

and 

Stimulated  Post- 
Search  Interview 

search 

Researcher 

Transcripts  of  the  think-aloud  protocolst  as  well  as  the  video-taped  search  sessions,  were  generated.  Think-aloud 
protocol  transcripts  provided  a verbative  translation  of  the  audio  tape.  Transcription  of  the  video  tapes  involved  a step- 
by-step  recounting  of  the  participant's  movement  through  the  information  space.  The  information  gathered  from  the 
video  tapes  was  transformed  into  a search  trail  generated  using  the  program  Inspiration®.  Locations  visited  by  the 
participant  during  the  search  are  represented  as  nodes  in  the  map. 

Preliminary  analysis  took  place  as  transcriptions  were  generated.  A participant  summary  sheet  was  used  to 
summarize  this  analysis  and  to  assist  in  organizing  the  data  (McGregor,  1993;  Miles  & Huberman,  1984).  The  summary 
sheet  addressed  each  research  question  according  to  the  instrument  examined,  directing  the  researcher  to  summarize  the 
main  points  from  the  survey  in  relation  to:  metacognitive  knowledge,  perceived  self-efficacy,  system  knowledge,  and 
subject  knowledge. 

Stimulated  post-search  interviews  were  scheduled  one  week  after  completion  of  dam  collection.  Initial  analysis 
of  the  think-aloud  protocols  and  search  trails  helped  to  guide  the  formulation  of  additional  questions  appropriate  for 
individual  interviews.  Video  tapes  were  shown  during  the  interview  to  stimulate  the  participant's  recall  of  particular 
instances  in  the  search  process. 

Data  were  examined  with  an  orientation  toward  both  breadth  and  depth.  In  order  to  obtain  breadth  and  depth,  the 
initial  pool  of  14  participants  was  reduced  to  four  for  in-depth  analysis.  The  researcher  reviewed  the  participants'  data  in 
relation  to  the  research  questions,  the  performance  of  each  participant  in  the  think-aloud  process,  and  the  need  to  represent 
the  range  of  participants  in  the  study.  The  four  participants  chosen  for  in-depth  analysis  were  Mick,  Bill,  Marsha,  and 
Alyssa.  As  shown  in  Table  2,  each  reflected  different  estimates  of  the  various  knowledge  described  previously.  The  high, 
mid,  and  low  designations  for  the  participants  reflects  an  overall  ranking  (“average")  based  on  level  of  disorientation, 
perceived  self-efficacy,  and  self-reported  knowledge  in  the  areas  of  metacognition,  system  and  subject.  Names  of  the 
participants  were  changed  to  protect  their  identify. 


Table  2.  Summary  of  participants  reported  knowledge, 
overall  ranking,  and  “success” 
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In-Depth  Analysis 

Given  the  variety  and  volume  of  evidence  gathered,  techniques  of  examination,  categorization,  tabulation  and  re- 
combination were  used  throughout  the  analysis.  Copies  of  original  data  sets,  as  well  as  records  of  each  transformation, 
were  used  to  maintain  the  integrity  of  the  data  during  analysis  iterations.  The  data  were  reviewed  and  triangulated  several 
times  by  the  researcher;  each  time  the  data  were  examined,  additional  relevant  information  and  related  findings  were 
revealed. 

The  initial  in-depth  analysis  was  driven  by  the  need  to  combine  the  think-aloud  protocols  and  the  search  trails. 

A “script**  format,  often  used  in  screen  plays,  movies,  and  television  was  used  to  merge  the  elements.  The  scripts  were 
used  throughout  the  remainder  of  the  analysis  to  aid  the  researcher  in  identifying  themes,  patterns,  and  strategies. 

The  next  round  of  analysis  involved  several  stages,  including  reading  through  the  data,  highlighting  instances  in 
the  data  related  to  the  research  questions,  and  generating  case  reports  for  individual  participants  (Bogdan  & Biklen,  1992; 
Harmon,  1992;  Yin,  1994).  “Reading  through  the  data**  involved  a review  of  all  data  elements,  “reading**  the  instruments 
like  a book.  Initial  themes  and  patterns  were  identified. 

/ Data  were  marked-up  (segmented)  in  accordance  with  the  research  questions  during  the  next  phase  of  analysis.  As 

the  researcher  read  the  data,  multi-coloured  highlighters  were  used  to  mark-up  the  data  according  to  research  questions  (see, 
Ericsson  & Simon,  1984,  for  a description  of  the  mark-up  process). 

The  sections  of  the  scripts  related  to  specific  research  questions  were  then  coded.  A mixture  of  established  and 
data-driven  codes  were  used  during  this  stage.  The  “established**  codes  were  derived  from  other  information  seeking 
studies,  including  information-processing  research,  information-seeking  models,  and  interactive  strategy  research.  These 
codes  relate  to  strategies  and  actions  taken  by  users  of  information  systems.  Themes  and  patterns  not  readily  applicable 
to  established  categories  were  assigned  a new  category. 

At  this  point,  the  researcher  asked  participants  to  confirm  the  validity  of  themes,  patterns,  and  strategies  (Bogdan 
& Biklen,  1992;  Mathison,  1988).  Inclusion  of  the  participant  in  the  research  process  is  a fundamental  element  in 
qualitative  research.  To  facilitate  feedback  from  each  participant,  individual  case  reports  were  generated.  Each  participant 
was  asked  to  review  the  report  and  to  generate  questions  or  concerns  for  discussion  with  the  researcher.  Verification  from 
the  participant  is  valuable  in  improving  the  validity  and  reliability  of  a study. 

One  week  after  the  reports  were  distributed,  the  researcher  met  with  each  participant  to  discuss  the  case  reports. 
Results  of  these  meetings  indicated  a high  degree  of  researcher-participant  agreement.  Recommendations  from  each 
participant  were  noted  during  the  interviews;  where  indicated,  changes  and/or  additions  were  made  to  the  data  sets  for  each 
participant. 

The  final  stage  of  analysis  involved  the  collation  of  data  according  to  the  corresponding  research  questions.  Each 
data  source,  organized  according  to  research  questions,  was  re-analyzed  to  match  and  clarify  patterns.  Pattern  matching 
involves  examination  for  similarities  in  the  thoughts  and/or  actions  of  the  participants.  Codes  were  assigned  to  the 
strategies  in  the  transcribed  protocol.  Examination  of  the  coded  protocol  was  undertaken  to  identify  trends  in  the  data. 

This  process,  repeated  for  each  participant  as  well  as  across  participants,  enabled  the  researcher  to  identify  trends  in 
strategies  as  they  related  to  each  research  question. 

In  order  to  interpret  the  meaning  cf  participants*  actions,  the  technique  of  explanation  building  was  used. 
Explanation  building  involves  returning  to  the  data  to  seek  explanation  for  why  participants  thought  or  acted  during  the 
search  process.  Returning  to  the  data  for  explanation  helped  to  establish  the  creation  of  assertions  related  to  the  study. 

Analysis  of  global  patterns  and  trends  was  necessary  to  address  the  overall  research  question.  To  assist  the 
researcher  in  theorizing  on  a global  level,  two  methods  of  aggregation  were  used  (Ericsson  & Simon,  1984):  aggregation 
by  episode  and  aggregation  by  process.  In  aggregating  the  data,  the  researcher  merges  individual  cases  into  one  instance. 
In  this  study,  aggregation  was  completed  around  two  data  points:  similar  episodes  in  each  search  and  similar  processes 
used  by  each  learner. 

The  final  stage  of  the  analysis  process  involved  cumulative  data  analysis,  where  all  trends,  issues,  and  themes 
generated  during  the  analysis  are  examined  as  complete  units  relating  to  each  research  question.  From  these  data,  the 
researcher  generated  speculations  and  explanations  of  the  strategies  used  by  learners  as  they  search  in  hypermedia 
information  systems. 


FINDINGS  AND  RESULTS 

The  results  of  the  study  were  consttucted  from  the  theoretical  foundations  of  grounded  theory.  Grounded  theory 
involves  building  explanations  from  the  data  themselves  (Glaser  & Strauss,  1967).  Pattern  matching  techniques  were 
employed  during  the  iterative  analysis  process  as  a way  of  making  sense  of  the  data.  Analysis  of  the  data  yielded  several 
assertions. 
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Assertions  By  Research  Area 

Assertion  1 : Perceptions  of  disorientation  affect  the  strategies  employed  in  a hypermedia  information  system. 
Several  learners  reported  feeling  “lost,”  or  “being  in  the  middle  of  nowhere”  when  they  were  using  the  system.  This 
disorientation,  in  tum,  affected  search  decisions. 

Assertion  2:  Prior  subject  knowledge  is  directly  related  to  concept  recognition  and  affects  the  strategies 
employed.  Learners  with  high  subject  knowledge  readily  recognized  terms  related  to  the  search  topic.  This  impacted  what 
they  pursued  in  the  system  and  how  they  used  the  program. 

Assertion  3:  System  knowledge  has  direct  impact  on  the  strategies  the  learner  is  able  to  develop  as  they  seek 
information  in  a hypermedia  information  system.  What  the  user  is  able  to  do  with  the  system  is  direcdy  tied  to  their 
knowledge  and  experience  with  the  system.  Development  of  strategies  takes  time;  learners  with  higher  system 
knowledge  were  able  to  develop  strategies  easier  and  quicker. 

Assertion  4:  Level  of  metacognitive  knowledge  influences  strategy  refinement.  Learners  with  high  levels  of 
metacognitive  knowledge  were  able  to  reflect  on  their  process.  This  reflection  enabled  them  to  refine  their  actions  and 
make  better  use  of  the  system. 

Assertion  5;  Self-efficacy  affects  willingness  to  engage  in  hypermedia  exploration.  Learners  with  high  levels  of 
self-efficacy  engaged  in  more  exploration  of  the  system.  This  increased  exploration  afforded  more  opportunities, 
increasing  prospects  of  finding  the  desired  information. 

Results  By  Research  Questions 

Overall  Research  Question:  What  strategies  are  employed  in  open-ended  hypermedia  information  systems? 

This  “meta-question”  was  informed  by  the  secondary  questions.  However,  the  findings  can  be  summarized  into 

three  trends: 

• A variety  of  strategies  are  used  by  learners  as  they  seek  information  in  a hypermedia  information 
system. 

• Metacognition,  system  and  subject  knowledge  affect  the  strategies  used. 

• Perceptions  of  disorientation,  as  well  as  perceived  self-efficacy,  affect  the  strategies  used. 

Specific  findings  are  presented  as  they  relate  to  each  research  question. 

Research  Question  One:  Does  metacognitive  knowledge  affect  the  strategies  employed? 

Of  the  prior  knowledge  areas  examined,  metacognitive  knowledge  most  influenced  the  strategies  used  while 
searching  using  Netscape®.  Level  of  metacognitive  knowledge  seemed  to  corresponded  to  success  in  system  use. 

Learners  who  used  the  system  effectively  were  active  information  processors  and  comprehenders  who  monitored  their 
learning  activities  (Osman  & Hannafin,  1992).  They  also  experienced  less  disorientation.  Mick  consistently  monitored 
his  thinking  and  revised  his  actions  accordingly,  and  also  made  significant  progress  with  the  strategies  used  and 
understanding  of  the  search  process.  Mick  engaged  in  several  metacognitive  tasks  described  by  Wang,  Haertel,  and 
Walberg  (1990):  comprehension,  monitoring,  use  of  self-regulation,  self-control  strategies,  and  use  of  the  strategies  to 
facilitate  generalization. 

All  learners  reported  metacognitive  experiences  during  their  search  tasks.  According  to  Flavell  (1979),  a 
metacognitive  experience  is  any  conscious  cognitive  experience  that  accompanies  and  pertains  to  an  intellectual 
enterprise.  His  illustration  of  a metacognitive  experience  directly  relates  to  what  the  participants  experienced  while 
searching  using  Netscape®:  ‘To  illustrate,  you  may  experience  a momentary  sense  of  puzzlement  that  you  subsequently 
ignore,  or  you  may  wonder  for  some  time  if  you  really  understand  what  another  person  is  up  to”  (p.  908).  Both  the  sense 
of  puzzlement  and  wondering  about  understanding  were  experienced  by  all  of  the  learners  in  the  study. 

Interestingly,  the  manner  in  which  metacognitive  experiences  were  processed  influenced  whether,  or  how, 
metacognition  influenced  the  search  task: 

Some  metacognitive  experiences  are  best  described  as  items  of  metacognitive  knowledge  that  have  entered 
consciousness.  As  one  example,  while  wrestling  with  some  stubborn  problem,  you  suddenly  recall  another 
problem  very  like  it  that  you  solved  thus  and  so.  Some  metacognitive  experiences  clearly  cannot  be 
described  that  way,  however.  For  instance,  the  feeling  that  you  are  still  far  from  your  goal  is  not  in  itself  a 
segment  of  metacognitive  knowledge,  although  what  you  make  of  that  feeling  and  what  you  do  about  it 
would  undoubtedly  be  informed  and  guided  by  your  metacognitive  knowledge...  (Flavell,  p.  908). 
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This  differentiated  Mick  from  the  other  learners:  not  only  did  he  have  prior  experience,  his  actions  were  informed  and 
guided  by  his  high  level  of  metacognitive  knowledge. 

While  the  importance  of  metacognitive  knowledge  is  clear,  the  implications  are  not.  Some  advocate  teaching 
metacognition  knowledge  for  skill  (Osman  & Hannafin,  1992;  Wang,  Haertel,  & Walberg,  1990);  others  have  trained 
students  to  effectively  monitor  their  learning  (Brown  & Palincsar,  1989;  Paris,  Cross,  & Lipson,  1984).  Still,  the 
results  are  inconsistent;  recommendations  for  teaching  or  training  are  often  suspect.  While  results  of  this  study  confirm 
its  importance,  further  investigation  is  needed  to  determine  the  best  ways  to  instruct  learners  in  the  use  of  these  skills. 

Research  Question  Two:  Are  perceptions  of  disorientation  influenced  by  the  strategies  employed? 

The  perception  of  disorientation  yielded  one  of  the  strongest  influences  on  the  strategies  employed.  High  levels 
of  disorientation  can  lead  to  dysfunction  as  well  as  overall  dissatisfaction  with  the  search  process.  This  is  not  only 
debilitating,  but  also  serves  to  elevate  frustration. 

Alyssa’s  high  level  of  disorientation  and  dissatisfaction  with  the  search  process  provides  the  most  compelling 
evidence  for  this  conclusion.  Her  failure  to  retrieve  relevant  information,  in  addition  to  her  use  of  low-end,  basic 
strategies,  suggests  disorientation  has  a seriously  detrimental  influence  in  the  use  of  emerging  information  systems. 

This  study  provided  more  evidence  for  the  phenomenon:  “lost  in  hyperspace.”  This  occurred  independent  of  the 
level  of  reported  knowledge.  Several  researchers  have  discussed  its  negative  consequences  on  usability  (see,  for  example, 
Jonassen  & Grabinger,  1989;  Marchionini,  1988).  Newby  (1992)  proposed  a way  to  help  orient  the  user,  reducing  the 
perceptions  of  disorientation:  involve  the  user  directly  in  the  retrieval  process.  Newby’s  “navigable  systems”  open  up 
the  black  box,  taking  the  user  “behind  the  scenes”  instead  of  leaving  them  “in  front  of  the  curtain,”  waiting  for  the 
system  to  return  results.  According  to  Newby,  easily  navigable  systems  enable  the  user  to  directly  engage  in  the  retrieval 
process,  which,  in  turn,  reduces  the  level  of  disorientation. 

Marchionini’s  (1993)  techniques  of  zooming  and  browsing  are  similar  to  Newby’s  navigable  system  space. 

Like  navigable  systems,  zooming  and  browsing  involve  the  user  directly  in  the  retrieval  process:  the  system  zooms  into 
a relevant  area  of  the  information  space,  offering  several  suggestions  that  may  or  may  not  be  relevant  to  the  learner’s 
needs.  After  moving  closer,  the  learner  is  better  able  to  browse  and  select  the  desired  information  from  what  is  available. 
The  system  does  not  supplant  entirely  the  effort  of  the  learner,  but  instead  engages  them  directly  in  the  process  of 
determining  relevance. 

In  some  ways,  Netscape®  provides  zooming  and  browsing  capabilities  for  the  World  Wide  Web.  The  learner 
submits  a request,  after  which  the  search  engine  scans  the  Web  information  space  and  returns  a list  of  “hits.”  At  this 
point,  the  concepts  of  zooming  and  browsing  break  down  in  several  important  ways.  A primary  way  that  it  breaks  down 
is  in  what  is  returned.  As  currently  structured,  the  listing  of  relevant  documents  can  range  from  titles  of  the  documents 
to  http://  addresses  to  ftp  sftes.  Obviously,  this  makes  the  list  less  intuitive  and  transparent. 

One  way  to  overcome  this  problem  might  be  to  standardize  how  the  systems  provide  feedback  to  the  learners. 
Several  systems  have  already  been  developed,  such  as  the  standardized  retrieval  scheme  developed  by  the  Library  of 
Congress,  which  has  been  tested  and  evaluated  for  several  years.  Standardized  searching  on  the  Web,  adapting  the  LC 
retrieval  model,  might  help  to  reduce  disorientation. 

Marchionini’s  (1993)  techniques  of  zooming  and  browsing  also  lose  validity  in  Netscape®  in  how  information 
is  ultimately  presented  to  users.  Zooming  and  browsing  can  invoke  images  of  movement  in  a three-dimensional  space. 
As  Netscape®  is  currently  designed,  a space  that  may  be  perceived  as  three  dimensional  is  flattened  to  two  dimensions. 
The  information  space  is  represented  on  a flat  screen  without  elements  of  depth. 

This  potential  “warping”  of  the  learner’s  mental  model  can  be  very  disorienting,  leading  the  learner  to  look  for 
“clues”  in  the  environment  to  re-orient  themselves.  Alyssa  looked  to  a map  to  improve  her  orientation  to  the 
information  available.  This  coincides  with  Forsyth’s  (1988)  recommendation  of  a “multi-sensory”  approach  for 
facilitating  memory  — as  well  as  improving  user  orientation. 

Severn, 1 options  are  available,  but  the  theoretical  “best”  option  remains  elusive.  It  is  clear,  however,  that 
guidelines  need  to  be  established  if  we  are  to  optimize  use  of  emerging  information  systems  (Jones,  1993).  The 
establishment  of  guidelines,  and  maintenance  within  certain  parameters,  is  crucial  if  learners  are  to  establish  useful 
mental  models  that  assist  them  in  their  understanding  (Norman,  1983).  Until  learners  are  able  to  engage  in  natural 
mapping  of  these  information  systems  (Norman,  1988),  and  guidelines  are  established  that  ensure  consistency  and 
predictability  (Wadlow,  1990),  disorientation  will  persist. 


279 


Research  Question  Three:  Does  perceived  self-efficacy  affect  the  strategies  employed? 

Perceived  self-efficacy  affected  both  the  number  as  well  as  the  types  of  strategies  which  were  engaged.  Those 
who  reported  mid-to-high  perceived  self-efficacy  (Bill,  Marsha:  mid;  Mick:  high)  engaged  in  more  strategies  and  at  a 
higher  level  than  Alyssa  who  reported  low  perceived  self-efficacy.  This  is  consistent  with  studies  examining  the 
relationship  between  self-efficacy  and  performance  (Ashton,  1984;  Kinzie  & Delcourt,  1991;  Murphy  et  al.,  1988),  where 
high  positive  correlations  were  reported  between  self-efficacy  and  performance. 

Perceived  self-efficacy  not  only  affects  the  learner’s  interactions,  but  also  perceptions  of  control.  System 
interactions  with  Bill  and  Mick  confirm  this  finding;  both  had  higher  levels  of  perceived  self-efficacy  and  were  successful 
in  their  task.  Although  Marsha  was  not  successful  in  her  target  task,  she  was  able  to  find  useful  information;  her 
confidence  and  high  perceived  self-efficacy  helped  elicit  this  result.  Alyssa,  in  contrast  to  the  others,  demonstrated  lower 
perceived  self-efficacy  throughout.  This,  in  turn,  influenced  her  interactions  and  strategies,  leaving  her  unsure  of  where 
she  was  or  how  to  proceed,  and  unable  to  move  beyond  basic  search  strategies. 

Research  Question  Four:  Does  system  knowledge  affect  the  strategies  employed? 

System  knowledge  also  affects  the  strategies  used  in  a hypermedia  information  system.  Participants  with  low 
system  knowledge  engaged  in  more  basic  search  strategies  than  those  with  higher  system  knowledge. 

The  level  of  system  knowledge  had  a stronger  influence  on  strategies  used  than  subject  knowledge.  This  is 
consistent  with  Park  and  Hannafin’s  (1992)  conclusion  that  the  lack  of  a functional  mental  model  of  the  multimedia 
system  tends  to  minimize  the  value  of  domain  knowledge.  While  Alyssa  possessed  some  domain  knowledge,  she  had  no 
prior  system  knowledge.  The  absence  of  a functional  mental  model,  as  well  as  an  inability  to  establish  one,  resulted  in 
very  limited  success  in  working  with  the  system  and  an  inability  to  move  beyond  basic  search  strategies.  While 
increased  system  knowledge  may  not  directly  lead  to  success,  system  understanding  is  critical  to  selecting  terms  and 
knowing  how  to  interact  Increased  system  knowledge  can  also  reduce  disorientation  and  frustration. 

Closely  related  to  a subject  being  searched  is  the  intent  behind  what  is  sought  in  the  system.  The  way  that 
World  Wide  Web  technologies  are  currently  designed  makes  the  environment  one  of  “at  your  own  risk.”  Anyone  can 
place  any  kind  of  information  on  the  Web.  In  addition,  there  is  minimal  structure  to  the  documents.  This  leaves  the  user 
in  a position  where  their  intent  may  not  match  that  of  the  document  retrieved.  A “best”  solution  for  this  issue  remains 
unresolved. 

The  ability  to  use  the  system  influenced  the  strategies  the  participants  engaged.  All  participants  entered  the 
search  task  with  no  prior  system  knowledge  directly  related  to  Netscape®.  While  Mick  reported  using  other  hypertext 
systems,  none  had  used  Netscape®  prior  to  its  introduction  in  class.  The  two  participants  who  built  a functional  model 
for  the  system,  Bill  and  Mick,  were  also  successful  in  their  tasks  and  moved  beyond  basic  search  strategies. 

Bill  and  Mick  were  also  the  only  participants  able  to  describe  and  depict  their  mental  model  of  the  system. 
Mayer’s  (1989)  work  on  conceptual  models  provides  an  interesting  interpretative  background.  Mayer  found  that  in  studies 
where  conceptual  models  were  provided,  learners  were  able  to  make  significant  strides  in  problem  solving  and 
understanding.  While  participants  in  this  study  were  not  provided  with  a model  prior  to  the  search  task,  those  who 
generated  their  own  conceptual  models  engaged  in  strategies  at  a problem-solving  level  and  increased  their  understanding. 

Consistent  with  previous  research,  increased  system  experience  and  system  knowledge  seems  to  affect  the 
strategies  used  in  the  system  (Weil,  Rosen,  & Wugalter,  1990).  The  participants  reinforced  this  observation  through 
their  interactions,  and  in  the  stimulated  post-search  interviews.  All  stated  that,  given  more  time  and  experience,  they 
were  confident  that  they  would  be  better  able  to  use  the  system. 

Research  Question  Five:  Does  subject  knowledge  affect  the  strategies  employed? 

The  level  of  subject  knowledge  affected  the  strategies  used  in  the  hypermedia  information  system.  While  the 
participants  used  a variety  of  strategies  independent  of  their  prior  subject  knowledge,  those  with  lower  subject  knowledge 
engaged  in  more  primary  search  strategies.  These  results  assist  in  confirming  what  other  researchers  have  found:  learners 
with  extensive  context-related  prior  knowledge  out-perform  their  cohorts  with  limited  prior  knowledge  (Langer  & 
Nicolish,  1981;  Recht  & Leslie,  1988).  Osman’s  (1992)  conclusion  that  the  availability  of  content-related  prior 
knowledge  as  a powerful  determinant  of  retrieved  information  was  also  supported  in  this  study. 

The  level  of  prior  subject  knowledge  also  affects  the  ability  to  integrate  and  retain  new  information  (Ausubel, 
1963).  Bill,  who  possessed  a higher  level  of  subject  knowledge,  refined  his  search  based  on  terms  he  previously  tried. 
Marsha,  who  possessed  little  prior  subject  knowledge,  refined  her  search  little,  using  few  terms  to  seek  information. 
While  the  number  of  terms  used  during  the  search  process  was  not  a strong  indicator  of  failure,  it  could  be  an  indication 
of  the  inability  to  monitor  and  judge  relevance  in  relation  to  information  sought.  According  to  Ausubel,  the  ability  to 
integrate  and  transform  information  is  a crucial  component  to  performance.  The  lack  of  integration  of  the  information 
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the  user  receives  while  searching  for  information  in  Netscape®  would  cause  conflict  in  relation  to  performance  and  search 
success. 


RECOMMENDATIONS  AND  IMPLICATIONS 

The  implications  for  creating  productive  and  successful  OELEs  are  compelling.  Several  studies  have  been 
conducted  indicating  the  promise  of  these  systems  (see,  for  example,  Harmon,  1992).  However,  much  work  is  needed  to 
realize  the  potential  of  OELEs  for  learning. 

One  area  that  was  a significant  influence  on  the  learner’s  successful  use  of  the  system  was  a feeling  of  insecurity 
or  discomfort.  Hannafin,  Hall,  Land  and  Hill  (1994)  discussed  this  as  a significant  issue  affecting  the  use  of  OELEs.  In 
the  present  study,  the  second  most  influential  factor  inhibiting  use  of  the  hypermedia  information  system  was 
disorientation,  which  led  to  discomfort  and  confusion.  The  methods  for  overcoming  disorientation  and  discomfort,  as 
well  as  for  assisting  and  supporting  learners  in  their  use  of  the  systems,  remains  unresolved. 

Another  issue  arising  from  this  study  relates  to  the  design  and  development  of  OELEs.  While  computer  and 
information  scientists  have  developed  design  models  for  creating  information  systems,  and  instructional  designers  and 
psychologists  have  created  design  models  for  the  development  of  learning  systems,  none  seem  to  satisfactorily  support 
the  design  and  development  of  OELEs.  It  is  an  area  rich  with  significant,  but  unfulfilled,  promise. 

The  need  for  guidelines,  both  in  terms  of  supporting  the  learner  and  in  developing  the  systems,  is  also  apparent. 
Guidelines  for  placing  information  on  the  Web  and  Internet,  as  well  as  indexing  and  providing  pointers  to  this 
information,  are  not  standardized.  This  lack  of  standardization  makes  the  development  of  these  systems  challenging  for 
the  designer,  and  also  presents  significant  challenges  to  users  of  the  systems  as  they  attempt  to  create  a functional  model 
(Norman,  1983)  of  the  information  system.  While  the  researcher  agrees  with  Jones  (1993)  that  “hard  and  fast”  rules  are 
neither  feasible  nor  desirable  for  these  environments,  guidelines  are  necessary  if  we  hope  to  see  the  continued  growth  and 
utility  of  these  environments. 

A final  implication  relates  to  how  individuals  think.  A fundamental  problem  underlying  the  lack  of 
understanding  associated  with  OELEs  is  the  manner  in  which  “compliant**  thinking  is  shaped  by  conventional  school 
activities  (McCaslin  & Good,  1992).  Traditionally,  education  has  been  a lock-step,  highly  directed  experience  (Hannafin, 
Hall,  Land,  & Hill,  1994).  This  engenders  learners  who  lack  the  orientation,  mental  models,  and  strategies  (or 
capabilities  for  creating  them)  for  these  environments  (Hill,  Lebow,  Driscoll,  & Rowley,  1994),  where  divergent 
thinking,  multiple  perspectives,  and  independent  thinking  are  critical. 

While  educational  history  was  not  gathered  in  the  current  study,  there  is  evidence  to  suggest  that  those  learners 
who  developed  an  orientation  and  functional  mental  model  of  the  system  proved  more  successful  in  their  task.  These 
learners  used  two  primary  techniques  associated  with  OELEs:  divergent  and  independent  thinking.  Adjusting  the  ways 
we  teach  to  foster  divergent  thinking  and  multiple  perspectives  may  assist  learners  in  using  these  environments 
effectively  and  with  minimal  disorientation. 

As  open-ended  hypermedia  information  systems  continue  to  grow,  both  in  their  capabilities  and  affordances,  the 
need  to  understand  how  to  support  learners  in  their  use  of  the  systems  becomes  critical.  The  promise  of  hypermedia  and 
OELEs  has  been  widely  heralded  (see,  for  example,  Bransford  et  al.,  1986;  Grabowski  & Small,  1991 ; Marchionini, 

1988;  Hannafin,  Hall,  Land  & Hill,  1994).  Continued  research  in  this  area  will  enable  the  realization  this  promise. 
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Whether  we  like  it  or  not  we  can  never  sever  our  links  with  the  past  complete  with  all  its  errors.  It  survives  in 
accepted  concepts,  in  the  presentation  of  problems  in  everyday  life,  as  well  as  the  language  and  institutions  that 
we  employ.  Concepts  are  not  spontaneously  created  or  generated  but  are  determined  by  prior  thought  - (Ludwik 
Fleck,  1979,  p.  9). 

This  paper  reports  on  some  of  the  theories  and  methods  that  could  charaterize  historical  study  in  the  field  of 
educational  technology.  It  will  begin  with  a discussion  of  the  importance  of  historical  study.  Then  it  willl  provide 
background  about  the  problems  facing  the  uses  of  differing  historical  methods  in  the  study  of  educational  technology.  A 
description  of  the  major  domains  of  historical  study  will  follow.  This  description  will  include  a deeper  discussion  of  the 
domains  of  intellectual  history,  including  arguments  for  the  use  of  two  specific  methodologies  in  the  study  of  the  history 
of  educational  technology,  the  history  of  culture  and  the  history  of  ideas. 

Introduction 

History  can  be  viewed  as  an  interaction.  It  can  be  the  interaction  between  the  empirical  and  the  hermenuetical  (the 
intent  to  understand),  the  interaction  between  science  and  art,  the  interaction  between  anaysis  and  expresion,  or  even  the 
interaction  between  events  and  thoughts. 

Many  professionals  seem  to  be  interested  in  the  history  of  the  field  in  which  they  practice.  Some  are  interested 
inknowing  their  intellectual  heritage,  others  would  merely  like  to  have  a sense  of  their  professional  roots. 

History  is  often  viewed  as  a sort  of  a body  of  information.  In  this  view  history  is  considered  to  be  a compilation  of 
data.  It  appeals  to  the  need  for  factual  content.  This  view  is  empirical. 

But  history  can  also  viewed  from  its  methodological  and  theoretical  starting  points.  In  this  view  history  is  guided 
by  the  methods  and  theories  that  are  followed  when  historical  studies  are  undertaken.  This  view  can  be  thought  of  as 
affective  because  the  attitudes  and  values  of  the  investigator  are  believed  to  effect  a study.  This  view  is  philosophical. 

These  two  views  of  history  do  not,  in  fact  could  not,  stand  in  isolation  from  each  other.  There  is  factual  content  in 
the  philosophical  view  of  history.  And  there  are  philosophical  considerations  in  the  empirical  view  of  history,  although 
most  often  these  are  assumed  and  therefore  the  resultant  reports  or  studies  are  considered  to  be  factual.  It  is  the  primary 
emphasis  of  each  of  these  viewpoints  that  contributes  to  their  different  outlooks  on  history. 

In  a field  such  as  educational  technology  where  each  person  formulates  her  or  his  of  view  of  what  the  field  is,  each 
person  could  have  a correspondingly  different  view  of  what  the  field's  history  is.  The  existance  of  a plurality  of  histories 
is  not  itself  problematic.  The  problem  arises  when  large  numbers  of  members  of  the  field  believe  that  only  certain 
factual  content  is  the  limit  of  the  study  of  the  history  of  the  field.  A discussion  of  the  theories  and  methods  of  historical 
study  can  identify  the  alternatives  that  individuals  can  adopt  or  adopt  for  their  own  specific  needs. 

Studying  the  history  of  educational  technology  should  help  to  answer  the  questions  like  'why  do  educational 
technologists  think  the  way  they  do  about  educational  technology?’  and  'how  has  that  way  of  thinking  changed  with 
time?’.  Perhaps  these  questions  appear  to  invite  mere  speculation  but  as  historian  George  Boas  stated,  "the  history  of 
ideas  tells  us  among  other  things  how  we  got  to  think  the  way  we  do-and  if  that  is  not  of  importance  one  wonders  what 
is”  (Boas,  1969,  p.  3). 

Why  study  history? 

History  is  strongly  connected  with  the  concept  of  change  in  two  ways.  First,  the  study  of  history  helps  to  show 
how  things  have  changed  and  helps  to  explain  why  they  are  the  way  they  are  at  present.  Second,  history  provides  the 
understanding  which  is  an  essential  component  of  reflective  thought.  The  study  of  history  can  help  maintain  traditions, 
help  individuals  to  stick  to  their  roots,  if  the  roots  are  valued  strongly  enough.  But  historical  analysis  may  also  provide 
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more  options.  An  historical  understanding  can  help  us  to  'break  out’  of  past  patterns  or  shift  emphases  if  it  seems 
important  to  do  so.  History  can  greatly  contribute  to  the  conscious  decisions  that  are  made  about  change. 

Robert  Duffy  (1988)  argues  that  "history  is  C;  itical  in  forging  a civilized  intellect"  (p.  460).  Duffy  suggests  that 
there  are  five  "desirable  experiences  and  habits  of  mind  uniquely  promoted"  (p.460)  when  studying  history;  (1) 
perspective,  (2)  encounters  in  cultural  literacy,  (3)  appreciation  of  relativism  in  a pluralistic  world,  (4)  analytical  and  (5) 
skeptical  habits  of  mind. 

Duffy  describes  perspective  as  "the  ability  to  see  people,  values,  ideas,  institutions,  or  events  against  a larger  convas 
of  antecedents,  related  situations  and  relevant  princilples"  (p.460).  He  argues  that  encounters  in  cultural  literacy  occur 
when  the  student's  values  and  ideas  confront  "great  ideas,  compelling  personalities,  events,  and  value  systems  of  the  past 
(p.461).  An  awarenes  of  relativism  arises  when  "students  are  emersed  in  other  cultures  at  other  times"  (p.461). 

Analytical  habits  are  improves  when  "contending  theses  are  permitted  to  sift  and  arrange  facts  into  differing  explanations 
of  the  event,  idea,  or  personality  at  issue"  (p.  461).  Skeptical  habits  involve  students  in  the  "need  for  data  and 
information  before  comitting  to  a proposition"  as  well  as  "a  quality  of  mind  that  can  live  with  tentativeness  where  the 
data  are  suggestive  but  insufficient"  (p.462). 

Harry  Helson  (1972)  argued  that  there  were  seven  reasons  for  psychologists  to  study  the  history  of  psychology. 

Since  much  of  the  field  of  educational  technology  is  so  closely  associated  with  psychology  I felt  that  it  might  be 
instructive  to  include  these  here.  These  seven  reasons  included:  (1)  "the  driving  power  of  ideas"  (p.  116),  that  ideas  stir 
emotions  and  lead  to  action;  (2)  "that  the  scientific  enterprise  is  more  or  less  continuous"  (p.l  16),  that  science  builds  on 
what  came  before  (even  if  it  were  wrong  or  had  to  be  changed);  (3)  "perspective  in  the  field  leads  one  to  discount  large 
claims"  (p.l  16),  that  attempts  have  always  been  made  to  claim  the  singular  importance  of  one  particular  method, 
approach,  or  device;  (4)  "outright  mistakes  may  be  avoided  if  one  sees  how  they  were  made  in  the  past  (p.l  16),  that 
while  history  may  not  be  cyclical,  patterns  do  emerge  and  occasionally  circumstances  sometimes  provide  the  opportunity 
to  avoid  repeting  errors;  (5)  to  locate  examples  for  teaching  and  scholarship;  (6)  "the  mere  joy  of  reading  historical 
accounts"  (p. ' ' 7),  which  emphasizes  the  aesthetic  dimension  of  both  reading  scholarly  works  and  the  satisfaction 
included  in  thi  struggle  of  academic  expression;  and  (7)  "that  success  as  a rule  is  not  achieved  immediately"  (p.  1 17)  in 
scholarly  thought. 

The  historiographical  problem  of  educational  technology 

Continuing  in  our  review  of  history  in  psychology,  John  Wettersten  (1975)  identified  some  fundemental 
historiography/  methodological/  theoretical  considerations  for  relating  the  history  of  scientific  psychology  The 
considerations  identified  by  Wettersten  are  also  important  to  studies  in  the  history  of  educational  technology.  This  is 
particularly  true  when  you  consider  how  many  educational  technologists  affiliate  themselves  with  psychology. 

Wettersten  identified  two  types  of  histories  : (1)  inductivist,  which  includes  true  theories,  facts,  the  discoverers,  and  dates 
of  discovery;  and  (2)  conventionalist,  which  says  that  the  theories  that  were  modified  to  get  to  present  theories  should  be 
included.  Wettersten  puts  the  fundemental  problem  of  writing  a history  of  scientific  psychology  this  way: 

"Though  historians  of  psychology  have  attempted  to  meet  both  the  inductivist  and  conventionalist  standards,  a 
successful  history  of  psychology  cannot  be  written  in  this  way.  There  is  one  crucial  fact  which  forces  historians  of 
psychology  (of  either  bent)  into  difficulties:  that  the  history  of  psychology  is,  for  the  most  part,  a history  of  schools. 
The  research  produced  by  the  schools  does  not  fit  inductivist  or  conventionalist  standards  because  the  theories  of  different 
schools  contradict  each  other.  Contradictory  theories  cannot  be  both  true,  or  both  be  modified  antecedents  of  any  unified 
contemporary  theory"  (p.157). 

To  some  extent,  attempts  to  write  a history  of  educational  technology  face  this  same  dilem  ..  Although  it  is 
certainly  true  that  the  nuances  of  practice  more  readily  allow  for  the  merging  of  differing  theory  bases.  But  there  are 
probably  more  interpretations  of  the  basic  theories  in  an  applied  field  which  further  complicates  the  problem. 

Wettersten  identified  five  techniques  which  were  used  by  historians  of  scientific  psychology  to  mask  their 
fundemental  problem.  These  techniques  "are  used  to  avoid  the  discussions  of  controversies,  mistakes  ans  problems,  and 
this  avoidance  leads  to  a misleading  picture  of  the  history  of  scientific  psychology"  (pp.  157-158).  The  five  techniques 
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are:  (1)  vague  and  uncritical  praise  of  theories;  (2)  recognition  of  fact  gathering  regardless  of  the  significance  of  the  facts; 

(3)  uncritical  praise  of  methodological  theory:  (4)  recognition  of  techniques  regardless  of  the  results  they  produce;  and  (5) 
discussion  of  careers. 

"How  can  the  history  of  psychology  be  portrayed  as  a steady  development  culminating  in  current  theory  when  any 
portrayal  of  current  theory  is  either  incomplete  or  inconsistant?"  (p.  158).  (because  it  does  not  account  for 
criticism/failure).  “There  are  two  solutions  to  the  problems  of  the  incompatibility  of  contemporary  theories:  one  may 
write  the  history  of  a single  school,  or  one  may  seek  to  reconcile  different  schools  and  their  histories"  (p.159).  (in  either 
casecriticism  and  failure  must  be  addressed). 

"Historians  of  scientific  psychology  have  intended  to  portray  the  steady  growth  of  the  application  of  the  scientific 
method  and  the  results  of  that  application.  This  may  be  interpreted  in  two  ways.  On  the  one  hand,  one  may  protray  (sp) 
the  intellectual  history  of  the  solution  to  problems  of  explaining  human  behavior.  On  the  other  hand,  one  may  relate  the 
story  of  the  growth  of  the  influence  and  study  of  scientific  psychology  as  a sociological  phenomenon"  (pp.  170- 171). 

This  is  analogous  to  the  history  of  ideas  and  cultural  history. 

"The  history  of  psychology  consists  for  the  most  part  of  the  development,  conflict  and  decline  of  schools... (if  critical 
history  is  avoided)... the  actual  problems,  theories  aims  and  mistakes  of  psychologists-  the  most  important  events  to 
understand  if  one  wants  to  understand  the  history  of  anv  intellectual  tradition-are  omitted  or  distorted"  (p.  171). 
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Proposal  Description 

This  project  involves  the  development  and  production  of  compact  disk-read  only  memory  (CD-ROM)  based  interactive 
media  for  the  recruitment  of  students  into  graduate  education  programs.  It  includes  the  incorporation  of  user  interfaces 
and  the  appropriate  hypermedia  linking  of  digital  video,  35mm  stills,  charts,  audio  and  textual  information  into  a CD- 
ROM  format.  Technical  production,  planned  dissemination,  and  user  feedback  analysis  are  also  addressed. 

Description 

Recruiting  by  colleges  and  universities  today  has  become  extremely  competitive.  To  maintain  a cutting  edge,  recruiters 
should  to  consider  technologies  beyond  those  that  are  paper-based  only.  While  successful  student  enlistment  in  this 
modem  day  era  is  still  accomplished  using  linear  technology  (mail  outs  of  brochures,  catalogs,  letters,  etc.)  high-tech 
alternatives  may  further  enhance  established  print  formats  and  generate  even  greater  success.  The  utilization  of  newer 
formats:  the  World  Wide  Web  pages  and  gopher  sites  on  the  Internet,  videotapes,  and  CD-  ROM  discs,  are  predicted  will 
pave  the  way  for  new  recruitment  directions.  Landoni  and  Catenazzi  (1993)  reported  that  there  were  many  disadvantages 
resulting  from  the  static  and  non-reactive  nature  of  paper-based  publications.  These  included. 

• difficulty  of  updating  the  content, 

• difficulty  of  locating  information, 

• absence  of  sound,  animation  and  moving  pictures, 

• cost  of  dissemination, 

• possibility  of  being  easily  damaged, 

• difficulty  of  customizing  information. 

Hypermedia  is  the  solution  to  all  of  the  above  described  problems.  It  may  prove  to  be  a good  alternative  to  the  more 
traditional  linear  formats.  Hypermedia  may  be  a tool  that  can  be  successfully  used  for  recruitment.  Grice  and  Ridgway 
(1993)  said  that  hypermedia  use  needs  to  be  judged  on  the  basis  of  how  well  it  helps  people  do  tasks.  They  recently 
stated  that: 

.hypermedia  is  here  today.  We  have  assumed  that  hypermedia  better  supports  the  same  tasks  linear  information 
supports.  ...  we  need  to  ask  whether  the  benefits  of  hypermedia  are  uniform,  or  whether  there  are  some  tasks  to 
which  hypermedia  is  better  suited.  This  is  an  area  where  research  would  be  valuable. 

Cortinovis  (1992)  additionally  supported  the  usage  of  the  computer  and  its  more  creative  aspects  to  develop  multimedia: 

It  is  only  in  a later  phase  that  the  new  technology  begins  to  be  creatively  exploited:  the  computer  is  used  to 
compose  and  read  hypermedia  documents.  ...the  computer  is  now  being  discovered  as  a “new”  medium:.. .it  can 
be  multimedia,  but  most  important  of  all,  it  is  interactive. 


Project 

The  objective  of  this  CD  -ROM  project  was  to  create  a master  and  multiple  copies  for  dissemination  of  information,  to 
potential  students  who  have  indicated  an  interest  in  our  graduate  and  certification  programs.  We  combined  pertinent 


information  from  the  following  sources: 

• our  graduate  catalog, 

• state  education  certification  bulletins, 

• parts  of  television  scripts  and  footage  used  in  recent  departmental  and  College  of  Education  productions, 

• photographic  stills,  slides  and  computer  generated  animation 

• portions  of  pre-recorded  video  taken  from  recent  interviews  that  were  conducted  with  current  graduates,  faculty,  and 


administrators  from  various  offices  on  campus. 

The  disk  contains  information  about  the  following  masters 
Education  (SHEd)  Department: 

• College  Teaching, 

• Community  College  Teaching, 

• Secondary  Education, 

• Vocational  Technical, 

• Educational  Microcomputing, 

• Educational  Media  and  Technology, 


and  doctoral  programs  offered  by  the  Secondary 
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• and  Library  Science. 

In  addition  to  information  on  the  various  academic  programs,  information  on  other  available  and  important  services  was 
provided.  These  included: 

• university  admissions 

• graduate  school  policies  and  requirements 

• availability  of  financial  aid  and  loans 

• a directory  of  SHEd  faculty  (including  e-mail  addresses) 

• community  demographics 

• health  and  medical  facilities, 

• public  schools  and  day  care  facilities 

• public  and  private  housing, 

• community  and  area  recreational  opportunities, 

• local  banking  and  loan  institutions 

• shopping  centers 

• extra-curricular  activities 

• accreditation 

Making  a CD-ROM  is  a process  that  costs  time  and  possibly  money,  but  a CD  can  provide  a powerful  medium  for  the 
transmission  of  information,  the  operation  of  programs,  or  the  storage  of  data.  The  huge  capacity  of  CDs  to 
accommodate  massive  amounts  of  data  and  large  programs  enables  them  to  be  used  for  the  dissemination  of  educational 
programs  in  a way  that  no  other  medium  can.  The  creation  and  production  of  CDs  has  previously  been  limited  to 
agencies  or  companies  which  could  afford  the  hardware  to  record  or  “bum”  the  CDs.  Recent  drops  in  the  cost  of  CD 
recorders  now  brings  the  process  of  CD  production  closer  to  users  at  large.  The  issue  of  prohibitive  time  constraints  for 
a CD  project  is  diffused  by  combining  the  talents  of  a team  in  a joint  venture,  drawing  on  the  special  abilities  of  each  of 
the  team  members.  The  process  is  straightforward,  however,  and  can  be  done  by  one  person  from  start  to  finish,  if  the 
scope  of  the  project  is  carefully  limited. 

The  hosedmll Steps, 

The  steps  in  the  process,  and  suggested  software  and  hardware  needed  to  complete  a CD  project,  are  delineated  below: 

1 . The  first  step  in  the  process  of  creating  a CD-ROM  is  creation  of  a flow-chart.  The  objectives  of  the  project  are 
translated  into  sections  and  sub-sections  for  the  program.  No  attempt  will  be  made  here  to  define  procedures  for 
making  a flow-chart.  This  will  vary,  according  to  project  requirements  and  program  parameters.  Beyond  the  flow 
chart,  creation  of  story-boards  can  be  used  to  embellish  and  flesh  out  the  program  content  (various  sub-sections  of 
the  flow-chart). 

2.  The  next  step  involves  the  collection  of  raw  materials  for  the  project.  This  includes  still  photos,  CDs,  videotapes, 
printed  documents,  and  the  like.  These  may  be  pre-fabricated,  even  commercial  (as  long  as  copyright  limitations  are 
honored),  or  they  may  be  tailor-made  by  the  CD  author.  Once  you  decide  to  make  a CD  and  have  completed  a flow 
chart  for  the  project,  you  must  either  find  or  create  the  components  that  will  form  the  content  of  your  program. 

3.  Still  photos  will  probably  be  scanned  into  files.  DeskScan  or  Omni-Page  Pro  can  be  used,  along  with  an  HP 
scanner  or  the  equivalent.  There  are  many  other  sources  for  pictures,  of  course,  such  as  CDs,  “digitized  camera” 
shots,  or  the  Internet.  Once  pictures  have  been  scanned,  they  can  be  edited  and  altered  in  a paint  program,  such  as 
Paint  Shop  Pro. 

4.  You  may  also  have  text  that  you  want  to  include  in  the  program  (more  than  likely);  an  easy  way  to  bring  text  into 
your  program  is  by  scanning.  Scanning  text  can  be  done  with  WordScan  or  with  such  software  as  Omni-Page  Pro. 
The  “Acquire”  feature  in  Word  6.0,  also  allows  you  to  scan  directly  into  a word  processing  text  file. 

5.  Video  utilizes  large  amounts  of  disk  space  in  a hurry.  It  will  need  to  be  used  judiciously,  always  with  a view  toward 
economical  treatment  of  this  medium.  Video  must  first  be  captured.  Digital  Video  Producer  Capture  (DVP)  was 
used  to  digitize  the  video  for  the  SHEd-CD  project.  We  used  a SVHS  player  to  advance  the  video  signal  to  the 
computer.  You  will  need  to  check  the  specifics  of  your  capture  card  as  they  vary.  Some  will  accept  multiple 
sources  of  video  while  others  are  more  narrow  in  their  input  capabilities.  The  DVP  program  allows  you  to  place 
your  clips  into  AVI  (video)  files  for  manipulation  within  the  CD  program. 

6.  Once  video  has  been  captured,  it  must  be  edited.  An  excellent  program  for  editing  video/audio  (motion)  and  stills,  is 
Adobe  Premiere  4.0.  This  software  allows  you  to  blend  your  clips  in  creative  ways.  It  also  provides  transitions  for 
scene  changes.  This  is  a powerful  editing  tool  and  is  quite  easy  to  use. 
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7.  All  of  these  components  will  need  to  be  brought  together  into  a meaningful  arrangement,  so  that  CD  users  will  get 
the  point  or  reach  the  objective  of  the  CD.  To  create  a user  interface  and  add  the  components  for  your  program,  you 
will  need  authoring  software.. .that  is,  unless  you  are  proficient  in  and  prefer  to  use  programming  language,  such  as 
C++.  The  SHEd-CD  project  was  created  in  an  excellent  authoring  program  called  Multimedia  Toolbook  (from 
Asymetrix  Corp.).  Learning  to  use  Toolbook  may  take  some  time,  but  its  ability  to  create  a framework  for  an 
interactive  CD  project  is  just  outstanding. 

8.  Finally,  the  CD  must  be  recorded  or  “burned”  on  a blank  disk.  Surprisingly,  CD  recorders  are  now  obtainable  for 
around  $1500,  and  will  become  even  more  affordable  in  the  immediate  future.  It  is  equivalent  to  making  a back-up 
cartridge  tape,  in  terms  of  difficulty  of  use...i.e.,  it  is  NOT  hard  to  do. 

Since  we  have  Pinnacle  CD-R  hardware  and  software  available  on  campus,  our  initial  time  delays  and  costs  were 
minimal.  Blank  CDs  can  be  purchased  for  $10-15  each.  After  recording  “one-off’  disks,  we  conducted  pilot  tests  to  see 
how  well  they  worked.  We  then  revamped  the  materials  where  needed,  and  burned  another  one,  until  we  were  satisfied 
with  the  product.  Kiosk  like  hyperlinks  to  various  topics  (via  buttons  and  icons)  were  used  as  the  primary  method  to 
access  the  menu  driven  information. 

In  order  to  determine  whether  the  product  was  “attractive”  to  students,  we  tested  it  on  graduate  students  from  our  campus. 
The  responses  have  been  exceptionally  positive.  Our  plan  is  to  replicate  and  distribute  the  disk  via  the  traditional 
methods  of  recruitment.  Because  replication  costs  are  dropping  rapidly,  they  no  longer  pose  a problem  in  terms  of  cost 
per  disk.  We  found  prices  in  the  $ 1.60/disk  range  for  500  discs  to  be  available  and  acceptable. 

The  SHEd-CD  project  is  evidence  that  CD  production  is  within  the  reach  of  many  who  may  believe  that  such  an 
achievement  is  best  left  with  experts  or  professionals.  You,  too,  can  make  a CD-ROM. 

Hardware  and  Software 

These  programs  assume  the  use  of  a Pentium  processor,  minimum  8 MB  RAM,  and-for  CD-ROM  creation-400  MB  or 
more  of  hard  drive  space.  Keep  in  mind  that  a CD  can  hold  up  to  around  680  MB,  which  means  that  you  may  make  your 
program  that  large  (especially  if  you  include  a lot  of  video),  in  which  case  you  will  need  a commensurate  amount  of  hard 
drive  space.  The  SHEd-CD  project  incorporated  the  use  of  the  following  software  programs: 

• Asymetrix  Multimedia  Toolbook  3.0  for  Windows 

• Digital  Video  Producer  Capture 

• Adobe  Premiere  4.0 

• Sound  Blaster  Audio 

• Video  for  Windows 

• Wave  Form  and  MIDI  sound 

• Paint  Shop  Pro 

• Desk  Scan  II 

• Word  Scan 

• Word  6.0 

• Pinnacle  CD-R 

• Morph 

• Animation  Works  
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Background 

Cooperative  learning  has  been  heavily  investigated  as  an  instructional  method  in  the  last  few  decades.  Johnson 
and  Johnson  (1991)  claim  that  when  students  learn  cooperatively,  they  learn  more,  like  school  and  each  other  more,  have 
higher  self-esteem,  and  have  improved  social  skills.  Slavin  (1983)  reported  that  cooperative  learning  has  a positive  effect 
on  student  self-esteem  liking  of  classes  and  classmates,  time-on-task,  cooperativeness  and  many  other  variables.  Other 
researchers  (Crooks,  Klein,  Jones,  & Dwyer,  1994;  Klein,  Erchul,  & Pridemore,  1994;  Klein  & Pridemore,  1992;  Snyder 
& Sullivan,  1995)  have  found  less  success  with  cooperative  learning  in  promoting  greater  student  achievement  and  better 

attitudes. 

Many  advocates  of  cooperative  learning  point  to  benefits  other  than  instructional  effectiveness  per  se  as  being 
sufficient  reason  to  use  cooperative  methods.  Several  researchers  have  found  students  who  work  cooperatively  had  better 
attitudes  toward  their  group  members  and  group  work  (Mevarech,  Stem,  & Levita,  1987),  liked  working  with  computers 
more  (Johnson,  Johnson,  & Stanne,  1986),  had  higher  goal  orientation  (Johnson,  Johnson,  & Stanne,  1985),  were  more 
satisfied  with  the  instruction,  and  had  higher  continuing  motivation  (Klein  & Pridemore,  1992) . Others  (Klein,  et  al„ 
1994;  Crooks,  et  al„  1994)  have  found  that  students  have  higher  continuing  motivation  for  computer-based  instruction 
when  working'alone'or  that  there  were  no  differences  in  attitudes  between  students  working  cooperatively  or  individually 
(Mevarech,  Silber,  & Fine,  1991;  Dalton,  Hannafm,  & Hooper,  1989).  The  mixed  results  from  cooperative  learning 
studies  indicate  that  there  is  still  much  to  be  learned  about  cooperative  learning. 

Several  researchers  have  suggested  that  personal  characteristics  and  preferences  of  the  learners  may  be  an 
important  factor  in  determining  the  effectiveness  of  cooperative  learning.  Chan  (1981)  and  Klein  and  Pridemore  (1992) 
have  studied  the  relationship  of  need  for  affiliation  to  performance  and  attitudes  under  cooperative  learning.  Slavin  (1983) 
indicated  that  a predisposition  to  cooperate  may  influence  the  occurrence  of  cooperative  behaviors. 

One  approach  that  could  conceivably  influence  the  effects  of  cooperative  learning  is  to  match  students  assessed 
preferences  for  cooperative  and  individual  learning  with  the  type  of  program  they  receive,  either  cooperative  or  individual. 
Freitag  and  Sullivan  (1995)  found  that  matching  learners  to  their  preferred  amount  of  instruction  increased  achievement 
and  produced  more  confidence  and  satisfaction  with  the  instruction.  Shute  and  Gluck  (1995)  reported  that  students  who 
exhibited  high  exploratory  behavior  in  an  instructional  program  performed  better  in  an  inductive  type  program  that 
matched  their  learning  style,  whereas  those  who  exhibited  low  exploratory  behavior  performed  better  in  a rule-based 
program  that  matched  their  learning  style.  It  seems  likely  that  matching  students  preferences  for  cooperative  and 
individual  learning  with  the  type  of  program  they  receive  could  also  maximize  their  potential  performance. 

The  purpose  of  this  study  was  to  compare  the  effects  of  assigning  students  to  a version  of  an  instructional 
program  (cooperative  learning  version  or  individual  version)  that  either  matched  or  did  not  match  their  pre-assessed 
preferences  for  cooperative  and  individual  learning.  Students’  preferences  for  cooperative  and  individual  learning  were 
assessed  two  weeks  prior  to  the  instruction  using  a Learning  Preference  Survey  created  for  this  study.  The  study 

permitted  investigation  of  the  following  research  questions. 

1 . Do  students  achieve  more  when  their  preference  for  group  work  is  matched  with  their  instructional  treatment 

than  those  who  receive  a contrasting  treatment? 

2.  Does  cooperative  or  individual  learning  yield  higher  achievement  scores  and  better  attitudes? 

3.  Does  matching  or  not  matching  student  preference  for  group  work  affect  interaction  patterns  within 
cooperative  dyads? 

Method 

Subjects  were  135  students  enrolled  in  an  urban  southwestern  high  school.  Data  from  104  students  were  used  in 
the  analyses.  Scores  from  31  subjects  were  discarded  due  to  absences  during  the  experiment.  Subjects  were  blocked  by 
preference  for  group  work  and  then  randomly  assigned  to  an  instructional  method  (cooperative  or  individual)  which  either 
matched  their  preference  (matched)  or  did  not  match  their  preference  (unmatched).  Achievement  was  measured  by  a 36- 
item  posttest  referenced  to  the  instructional  objectives  of  the  instructional  program,  and  attitude  was  measured  by  a 12- 
item  survey  that  assesses  liking  of  the  program,  liking  of  cooperative  learning,  effort,  and  continuing  motivation.  En- 
route  behaviors  were  tracked  by  the  computer,  and  student  interactions  recorded  by  trained  observers. 

All  students  were  given  instruction  on  how  to  work  cooperatively  and  directions  to  facilitate  cooperative 
learning  were  incorporated  into  the  cooperative  learning  version  of  the  instructional  program.  Each  student  was  also 
given  a Learning  Preference  Survey  to  determine  their  preference  for  group  work.  A median  split  was  used  to  classify 
students  as  having  a high  or  low  preference.  Students  were  then  assigned  to  an  instructional  method  (cooperative  or 
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individual)  which  either  matched  (matched)  or  did  not  match  (unmatched)  their  preference  in  a 2 x 2 (matching  condition  x 
instructional  method)  posttest  only  experimental  design. 

Students  spent  approximately  three  50-minute  class  periods  working  either  individually  or  in  pairs  to  complete  a 
CAI  lesson  about  basic  geometry  concepts  such  as  lines,  rays,  angle,  and  polygons.  Each  lesson  contained  information, 
examples,  exercises,  review,  and  practice  problems.  After  completing  the  CAI  lesson,  each  student  completed  the 
posttest  and  the  attitude  survey  individually. 

A Pearson  product-moment  correlation  coefficient  was  calculated  to  determine  whether  a significant  relationship 
existed  between  preference  for  group  work  and  student  ability  as  measured  by  IOWA  test  scores  for  Reading.  The  Pearson 
correlation  coefficient  of  -0.16,  pc.  16,  revealed  a non-significant  relationship.  Therefore,  it  was  not  deemed  necessary  to 
adjust  posttest  scores  for  difference  in  ability  between  students  with  a high  preference  for  group  work  and  students  with  a 
low  preference  for  group  work. 

The  posttest  data  were  analyzed  using  an  analysis  of  variance  (ANOVA).  A multivariate  analysis  of  variance 
(MANOVA)  was  used  to  analyze  the  student  attitude  data,  followed  by  univariate  analyses  for  each  question  on  the 
attitude  survey.  Time  in  program  was  recorded  and  analyzed  using  analysis  of  variance  (ANOVA).  Learning  efficiency 
ratios  were  computed  by  dividing  the  each  student’s  posttest  score  by  the  student’s  time  in-program  and  were  analyzed 
using  analysis  of  variance  (ANOVA).  Differences  in  student  in-program  behaviors  were  analyzed  using  chi-square.  All 
analyses  were  conducted  at  an  alpha  level  of  .05. 

Result? 

The  mean  scores  and  standard  deviations  for  achievement  by  preference  and  matching  condition  are  shown  in 
Table  1.  Students  in  the  unmatched  conditions  (M  = 21.77)  had  higher  significantly  achievement  scores  than  students  in 
the  matched  conditions  (M  = 19.15),  F(l,100)  = 5.04,  p < .05.  The  mean  achievement  score  for  students  with  a high 
preference  for  group  work  (M  = 20.50)  was  only  slightly  higher  than  for  students  with  a low  preference  (M  = 20.42),  a 
non-significant  difference.  The  preference  by  matching  condition  interaction  also  was  not  significant.  The  overall  mean 
achievement  score  was  20.46  or  57  percent. 

The  matching  condition  variable  yielded  an  effect  opposite  to  the  predicted  one  - that  is,  unmatched  subjects 
scored  significantly  higher  than  matched  ones,  thus  indicating  that  matching  subjects  to  their  preferred  condition  does  not 
produce  a positive  achievement  effect.  Therefore,  the  decision  was  made  to  j>erform  a second  2x2  analysis  of  variance 
for  achievement  using  learner  preference  and  the  actual  experimental  treatments  of  cooperative  and  individual  learning 
received  by  the  subjects,  rather  than  the  experimenter-derived  matched  and  unmatched  conditions.  Given  the  lack  of 
positive  predictive  value  of  the  matching  variable,  it  was  felt  that  analysis  using  the  actual  treatment  conditions  of 
cooperative  and  individual  learning  would  be  more  useful  for  understanding  and  explaining  the  results  than  those  using 
matching  condition. 

The  preference  (high,  low)  by  program  type  (cooperative,  individual)  data  are  reported  in  Table  2.  This  analysis 
resulted  in  re-arrangement  of  the  mean  scores  from  two  cells  in  Table  1 and,  consequently,  different  ANOVA  results. 
Subjects  under  cooperative  learning  had  a mean  score  of  21 .46,  and  those  under  individual  learning  had  a mean  of  19.46,  a 
non-significant  difference,  E(l,100)  = 2.96,  p < .09.  The  mean  scores  for  high-preference  subjects  (M  = 20.50)  and  low- 
preference  subjects  (M  = 20.42)  remained  the  same  as  in  the  initial  ANOVA,  of  course,  and  did  not  differ  significantly 
from  one  another.  However,  the  preference  by  program  type  interaction  was  statistically  significant,  F(  1,100)  = 5.04,  p 
< .05. 

The  significant  preference  by  program  type  interaction  reflected  the  fact  that  students  with  a low  preference  for 
group  work  scored  higher  (M  = 22.73)  when  they  worked  cooperatively  than  when  they  worked  individually  (M  = 18.11), 
whereas  students  with  a high  preference  for  group  work  scored  slightly  lower  when  working  cooperatively  (M  = 20. 19) 
than  when  working  individually  (M  = 20.81).  Tukey’s  post  hoc  method  for  pairwise  comparisons  revealed  that  the 
achievement  score  of  22.73  for  the  students  in  the  low  preference-cooperative  condition  was  significantly  higher  than  the 
achievement  score  of  1 8. 1 1 for  the  students  in  the  low  preference-individual  condition  at  the  .05  level.  All  other  pairwise 
comparisons  were  not  significant. 

Table  3 shows  the  mean  times  in  program  by  preference  and  program.  Students  with  a high  preference  for  group 
work  averaged  101 .45  minutes  to  complete  the  program,  and  those  with  a low  preference  for  group  work  averaged  80.35 
minutes,  a significant  difference,  E(l,58)  = 9.63,  p<  .01.  Students  in  the  cooperative  dyads  averaged  84.06  minutes  in 
the  program,  and  students  working  alone  averaged  95.94  minutes,  a nonsignificant  difference. 

The  groups  with  higher,  though  not  significantly  higher,  posttest  scores  for  each  variable  (low-preference  over 
high-preference,  cooperative  over  individual)  also  spent  less  time  in  the  program  than  their  counterparts.  Therefore, 
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learning  efficiency  ratios  (LERs)  were  computed  for  each  student  and  group  to  determine  whether  there  were  significant 
between-group  differences  in  learning  efficiency  - i.e.,  the  amount  learned,  as  measured  by  posttest  scores,  per  unit  of 
time.  Each  student’s  LER  was  calculated  by  dividing  his  or  her  posttest  score  by  his  or  her  time  in  program.  For 
example,  a student  with  a posttest  score  of  20  and  a time  in  program  of  80  minutes  would  have  a learning  efficiency  ratio 
of  .25  (20  + 80).  Table  4 shows  the  mean  learning  efficiency  ratios  by  preference  and  program  type. 

The  learning  efficiency  ratio  data  in  Tables  4 reveal  that  students  with  a low  preference  for  group  work  (LER  M 
= .30)  had  significantly  higher  LERs  than  students  with  a high  preference  for  group  work  (LER  M **  -22),  E0.79)  = 

7.81,  p < .01.  Students  in  the  cooperative  treatment  (LER  M = -29)  had  significantly  higher  LERs  than  those  in  the 
individual  treatment  (LER  M = .22),  F(1 ,79)  = 7.83,  p < .01 . The  preference  by  program  type  interaction  for  learning 
efficiency  ratios  was  not  statistically  significant. 

The  responses  to  a 12-item  attitude  questionnaire  administered  at  the  end  of  the  program  were  scored  on  a four 
point  scale  from  1 (strongly  agree)  to  4 (strongly  disagree).  MANOVA  revealed  a significant  difference  for  program  type, 
F(12,83)  = 4.23,  p < .001,  but  not  for  preference  or  the  preference  by  program  type  interaction.  Therefore,  item-by-item 
univariate  analyses  were  performed  only  for  program  type.  The  univariate  analyses  for  program  type  revealed  significant 
differences  for  three  statements.  Students  working  alone  agreed  significantly  more  strongly  than  students  working 
cooperatively  with  the  statement  “I  would  like  to  learn  more  about  geometry”,  E(1 ,94)  = 7.63,  p < .01 . Students  who 
worked  cooperatively  agreed  more  strongly  with  the  statement  “I  would  like  to  learn  new  math  concepts  with  a group”, 
E(l,94)  = 7.46,  p<  .01,  than  students  who  worked  individually  agreed  with  the  statement  I like  to  learn  new  math 
concepts  by  myself’.  Similarly,  students  who  worked  cooperatively  agreed  more  strongly  with  the  statement  “I  liked  ^ 
working  in  groups”,  E(l,94)  = 18.31,  p<  .001,  than  the  students  who  worked  individually  agreed  with  the  statement  “I 

liked  working  alone”. 

Chi-square  was  used  to  analyze  number  of  observer-recorded  cooperative  interactions  for  differences  between 
groups.  The  high-preference  cooperative  dyads  had  more  instances  of  each  of  the  four  cooperative  behaviors  and  of  off- 
task  behaviors  than  the  low-preference  pairs.  These  differences  were  significant  for  “asks  questions”  (99  instances  for 
high-preference  subjects  and  69  for  low-preference  subjects),  “discussion  of  lesson  content”  (203  instances  for  high- 
preference  and  150  for  low-preference),  and  “shares  mouse  or  keyboard”  (17  instances  for  high-preference  and  2 for  low- 
preference),  and  for  off-task  behaviors  (220  for  high-preference  subjects  and  94  for  low-preference  subjects). 

Discussion 

The  present  study  examined  the  effect  of  matching  student  preference  for  group  work  with  the  instructional 
treatment  that  students  received.  Students  with  a high  preference  for  group  work  were  assigned  to  work  either 
cooperatively  (matched)  or  individually  (unmatched),  and  conversely,  students  with  a low  preference  for  group  work  were 
assigned  similarly  (cooperatively  = unmatched  and  individually  = matched). 

It  was  expected  that  students  whose  preferences  for  group  work  were  matched,  rather  than  unmatched,  with 
instructional  treatment  would  score  higher  on  the  posttest.  In  fact,  when  students  were  assigned  to  an  instructional 
treatment  which  did  not  match  their  preference,  they  did  better.  Thus,  a preference  for  group  work  did  not  predict  better 
performance  under  group  work.  Rather,  the  data  on  interactions  and  off-task  behaviors  suggest  that  it  is  a predictor  of 
desire  to  interact  with  others  or  sociability.  Students  with  a high  preference  for  group  work  who  worked  cooperatively 
exhibited  significantly  more  off-task  behaviors,  such  as  talking  to  their  partner  or  members  of  other  groups  about  topics 
unrelated  to  the  computer  program,  than  those  with  a low  preference  for  group  work  who  worked  cooperatively. 

The  analysis  of  the  time  spent  in  the  program  also  seems  to  support  the  idea  that  the  preference  for  group  work 
measure  was  more  a predictor  of  sociability  than  of  performance.  Students  with  a low  preference  for  group  work,  in  fact, 
spent  less  time  in  the  program  than  their  high-preference  counterparts  with  no  significant  difference  in  performance.  The 
smaller  amount  of  time  spent  completing  the  program  and  the  fewer  number  of  off-task  behaviors  for  the  low-preference 
groups  may  indicate  a learning  approach  that  is  more  effective  because  it  is  more  focused  on  the  learning  content  and  less 
subject  to  interference  factors  resulting  from  the  off-task  behaviors. 

There  is  also  considerable  evidence  that  students  may  often  prefer  an  instructional  method  that  does  not  yield 
greater  achievement.  Peterson  and  Janicki  ( 1979)  found  that  elementary  students  who  expressed  a preference  for  small- 
group  or  large-group  work  had  higher  retention  scores  when  they  received  the  instructional  method  that  was  opposite  their 
preference.  Shute  and  Gluck  (1995)  found  that  if  students  had  been  assigned  to  treatments  by  their  stated  preference  for 
exploratory  behavior,  no  significant  difference  in  achievement  would  have  occurred  between  those  who  were  matched  to 
their  preference  and  those  who  were  not.  Similarly,  Hannafin  (1994)  found  that  students  who  indicated  a preference  for  a 
lean  instructional  program  had  better  achievement  scores  when  they  were  not  matched  to  their  preference. 
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There  is  a common  belief  in  educational  technology,  as  well  as  in  other  areas  of  education,  that  students  are 
good  judges  of  the  ways  they  will  learn  best.  This  belief  was  an  important  factor  in  the  design  of  the  present  study.  In 
retrospect,  however,  it  is  important  to  examine  why  this  may  not  be  the  case.  Students  may  not  be  able  to  readily 
identify  methods  which  will  increase  their  achievement  on  school  tasks  or  they  may  not  care  much  about  relatively  minor 
improvements  in  achievement.  Their  preferences  may  be  stronger  for  methods  which  they  think  will  require  less  work, 
be  more  fun,  allow  for  more  social  interaction,  or  give  them  more  freedom  of  choice. 

Although  the  cooperative  students  had  higher  posttest  scores,  they  did  not  differ  significantly  from  the  scores  of 
the  students  who  worked  alone.  Several  factors  may  account  for  the  fact  that  cooperative  learning  did  not  have  a stronger 
effect.  A number  of  recent  studies  have  shown  that  cooperative  learning  may  not  be  as  powerful  an  instructional  method 
as  previously  thought.  Snyder  and  Sullivan  (1995)  and  Klein  et  al.  (1994)  found  that  students  learning  alone  outscored 
students  who  worked  cooperatively.  Crooks,  et  al.  (1994),  Crooks  (1995),  and  Doran  (1994)  did  not  find  significant 
differences  in  achievement  between  students  who  worked  cooperatively  and  those  who  worked  alone.  With  the  exception 
of  Snyder  and  Sullivan  (1995),  all  of  the  above  research  was  conducted  with  college  students.  It  may  be  that  older 
students  do  not  benefit  from  cooperative  learning  as  much  as  younger  ones.  Most  of  the  research  which  has  shown 
achievement  differences  in  favor  of  cooperative  learning  (Johnson  et  al.,  1985,  1986;  Slavin,  1983)  has  been  conducted 
with  younger  students. 

Cooperative  learning  advocates  often  attribute  the  lack  of  a significant  difference  favoring  cooperative  learning  to 
the  students  not  being  sufficiently  accustomed  to  working  together  to  benefit  from  the  cooperative  learning  situation 
and/or  the  fact  that  cooperative  learning  conditions  do  not  include  Johnson  and  Johnson's  (1991)  five  key  elements. 

These  elements  are  positive  interdependence,  face-to-face  interaction,  individual  accountability  and  personal  responsibility, 
interpersonal  and  small  group  skills,  and  group  processing.  While  these  explanations  may  often  be  true,  it  also  is  often 
true  that  learners  do  not  have  good  skills  for  studying  individually  and  that  the  best  conditions  for  learning  often  are  not 
incorporated  into  individual  learning  treatments.  In  the  present  study  positive  interdependence,  individual  accountability 
and  personal  responsibility  were  intended  to  be  fostered  by  having  students  take  the  test  individually  and  receive  extra 
points  on  the  test  if  the  average  of  the  test  scores  of  the  partners  was  above  a certain  level.  The  incentive  of  a good  test 
grade  may  have  not  been  a sufficient  motivator  to  encourage  the  students  to  work  together.  Most  students  seemed  more 
inclined  just  to  get  through  the  program  than  to  learn  the  material  well. 

The  present  results  raise  the  possibility  that  students'  “sociability1  ‘ may  influence  the  effectiveness  and  efficiency 
of  cooperative  learning  as  an  instructional  treatment  for  them.  Research  that  investigates  the  relationship  between  a 
measure  of  sociability  and  achievement  under  cooperative  learning  could  incicate  the  extent  to  which  sociability  impairs 
or  facilitates  student  learning  in  cooperative  groups  and  may  have  implications  for  group  composition  in  cooperative 
learning  situations.  Research  of  this  type  may  help  us  better  understand  the  conditions  under  which  cooperative  learning 
is  most  effective  in  the  schools. 
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Tabic  1 


Mean  Posttest  Scores  by  Preference  and  Matching  Condition 


Matching  Condition 

High 

Preference 

Low 

Totals 

Matched 

20.19 

18.11 

19.15 

(6.40) 

(5.90) 

(6.18) 

Unmatched 

20.81 

22.73 

21.77 

(5.96) 

(5.45) 

(5.74) 

Totals 

20.50 

20.42 

20.46 

(6-13) 

(6.09) 

(6.08) 

Table  2 

Mean  Posttest  Scores  by  Preference  and  Program  Type 


Program  Type 

High 

Preference 

Low 

Totals 

Cooperative 

20.19 

22.73 

21.46 

(6.40) 

(5.45) 

(6.02) 

Individual 

20.81 

18.11 

19.46 

(5.96) 

(5.90) 

(6.03) 

Totals 

20.50 

20.42 

20.46 

(613) 

(6.09) 

(6.08) 

Table  3 

Mean  Total  Time  in  Minutes  bv  Preference  and  Program  Type 


Program  Type 

Preference 

High 

Low 

Totals 

Cooperative 

98.27 

68.43 

84.06 

(33.57) 

(18.67) 

(30.88) 

Individual 

102.96 

86.98 

95.94 

(24.46) 

(30  73) 

(28.19) 

Totals 

101.45 

80.35 

91.92 

(27.30) 

(28.16) 

(29.43) 
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Table  4 


Mean  Learning  Efficiency  Ratio  bv  Preference  and  Program  Type 


Preference 

Program  Type 

High 

Low 

Totals 

Cooperative 

0.24 

0.35 

0.29 

(0.14) 

(0.12) 

(0.15) 

Individual 

0.21 

0.24 

0.22 

(0.08) 

(0.11) 

(0.09) 

Totals 

0.22 

0.30 

0.26 

(0.11) 

(0.13) 
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ABSTRACT 


To  prevent  the  user  from  the  over-relying  on  technology  support,  we  propose  a 3-D  contingent  scaffolding 
model  to  systematically  vary  the  instructor’s  support  in  response  to  the  learner’s  performance  in  a learning  task 
consisting  of  a sequence  of  steps/sub-tasks.  We  report  on  the  implementation  of  this  model  in  a computer-based 
instruction  program  here  and  on  a study  of  its  effectiveness.  In  this  study,  the  3-D  contingent  scaffolding  model  is 
implemented  in  a computer-based  instruction  program,  “Hypothesis  Testing--  the  Z-Test”  in  order  to  establish  baseline 
data  for  integrated-media-based  instruction  or  hypermedia  based  learning  environment.  Our  findings  show  evidence  that 
“Hypothesis  Testing-  the  Z-Test"  does  indeed  promote  knowledge  maintenance  and  improve  independent  learning 

INTRODUCTION 

The  term  “Scaffolding”  has  been  used  to  describe  various  instructional  techniques  for  use  in  learning  activities 
that  reflect  authentic  task  situations.  This  technique  has  also  drawn  great  attention  from  the  media  researchers  for  media 
can  provide  a more  realistic  learning  environment  with  rich  and  varied  support.  One  major  concept  of  this  technique  is  to 
enable  the  learner  to  engage  in  out-of-reach  activities;  having  a “knowledgeable  other”  or  “more  capable  peer”  to  bring  the 
learner  along;  having  something  or  someone  “share  the  cognitive  load”  (Jackson  et  al„  1995).  However,  if  we  go  back 
to  Bruner’s  original  definition  of  scaffolding  (Bruner,  1983),  we  find  there  is  another  valuable  key  concept  of  this 
technique  that  has  usually  been  neglected.  That  is,  this  knowledgeable  or  more  capable  other  is  not  only  responsible  for 
building  the  support  in  learning  activities,  but  also  is  responsible  for  fading  the  support  as  the  learner  gradually  masters 
the  skill.  The  Cognition  and  Technology  Group  at  Vanderbilt  University  (1993)  has  proposed  a risk  of  over  relying  on 
the  support  of  integrated  media.  To  educate  individuals  to  be  independent  and  active  learners,  there  is  a need  to  emphasize 
the  support-fading  component  of  scaffolding  while  we  try  to  integrate  this  instructional  technique  into  any  kind  of  media 
assisted  instruction. 


BACKGROUND 
Method  of  Scaffolding 


In  the  past  20  years,  the  constructivist  paradigm  has  gradually  come  to  educational  research.  From  the 
constructivism  perspective,  the  purpose  of  education  is  to  cultivate  independent  and  self-directed  learners.  Bruner  s 
metaphor  of  scaffolding  provides  an  explicit  strategy  to  direct  our  teaching  toward  this  end. 

Scaffolding  refers  to  the  interactional  support  that  instructors  or  more  skillful  peers  offer  learners  to  bridge  the 
gap  between  their  current  skill  levels  and  a desired  skill  level.  In  the  process,  the  amount  of  support  is  gradually 
withdrawn  as  the  learners  become  more  proficient.  Ultimately,  the  learners  can  complete  tasks  on  their  own  (Greenfield, 
1984).  The  literature  review  suggests  that  scaffolding  can  enhance  comprehension,  improve  independent  learning,  and 
promote  knowledge  transfer  (Bruner,  1983,  Greenfield,  1984,  Cazden,  1988).  Evidence  of  these  advantages  has  been 
found  in  many  studies  in  the  field  of  language  and  cognitive  development  (Gallimore  et  al.,  1989:  Boyle  and  Peregoy, 
1990;  Day  and  Cordon,  1993). 

At  the  same  time,  limitations  of  scaffolding  have  also  been  pointed  out.  This  technique  has  been  criticized  for 
its  lack  of  discussion  on  development  of  the  expert’s  role  in  providing  the  novice  with  assistance  (Gaffney  and  Anderson, 
1991).  Its  implementation  has  also  been  criticized  for  not  being  able  to  capture  the  challenge  of  responding  to  the 
diversity  of  children’s  intentions  in  classroom  teaching  (Dyson,  1990). 

Scaffolding  in  Media-Based  Instruction 

Computers  have  introduced  unprecedented  levels  of  autonomy  into  education.  The  processing  and  integrating 
capabilities  of  computers  have  created  an  interactive,  support-rich,  and  individualized  learning  environment.  These 
characteristics  might  break  the  limitations  of  scaffolding  and  ease  the  implementation  of  this  instructional  technique. 


Several  researchers  have  developed  scaffolded  computer-based  instruction,  integrating  more  than  one  medium  to  support 
learners’  knowledge  construction  in  authentic  learning  activities  (Steiner  & Moher,  1994;  Jackson  et  al.,  1995). 

However,  the  discussion  has  been  focused  on  the  software  development  and  the  support-building  models.  Only  a small 
amount  of  qualitative  evaluation  data  have  been  reported.  The  support-fading  element  of  scaffolding  was  not  applied  in 
these  studies.  Therefore,  solid  scaffolded  instruction  is  not  only  difficult  to  built,  but  also  extremely  difficult  to  evaluate. 

Leiberman  and  Linn  (1991)  contented  that  scaffolding  is  one  of  the  several  ways  computers  can  be  used  to  encourage 
students  to  be  self-directed.  This  should  be  attributed  to  the  support-fading  skill  used  in  the  scaffolding.  However,  to  use 
this  skill,  the  major  challenge  will  be  when  the  support  should  be  faded  and  how  much  the  support  should  be  reduced  at 
the  time.  This  requires  us  to  find  out  another  technique  allowing  us  to  know  the  learner’s  mastery  level  at  any  point  of 
the  scaffolded  learning  process. 

LINKING  THE  CONCEPT  OF  SCAFFOLDING  TO  INTEGRATED-MEDIA  DESIGN  FEATURES 

In  this  session,  we  sought  for  an  operational  definition  of  scaffolding.  Based  on  the  definition,  we  proposed  a 3- 
D scaffolding  model  which  provides  a systematic  way  to  apply  scaffolding  in  integrated-media  based  instruction. 

The  Elements  of  Scaffolding  Relevant  to  Integrated  Media  Design 

To  link  the  concept  of  scaffolding  to  the  integrated-media  design  features,  it  is  important  to  identify  some  basic 
elements  of  scaffolded  instruction  which  are  especially  relevant  to  the  integrated-media  design.  To  illustrate  the  features 
of  those  elements,  the  metaphor  of  an  adult  aiding  a toddler  (Cazden,  1988)  will  be  used  as  an  example. 

1 . Hierarchical  component  skills 

In  the  metaphor  of  scaffolding,  the  acquisition  of  skill  is  conceived  as  a hierarchical  program  in  which 
component  skills  are  combined  into  “higher  skills”  by  appropriate  orchestration  to  meet  new,  more  complex  task 
requirements  (Bruner,  1973).  Therefore,  the  crucial  task  in  scaffolding  often  possesses  a variety  of  relevant 
components  (usually  in  an  appropriate  serial  order)  necessary  to  achieve  a particular  end.  And  that  is  the  instructor’s 
responsibility  to  decompose  the  Final  task  into  hierarchical  component  skills  based  on  the  nature  of  the  task  and  the 
learner’s  ability.  In  the  example  of  an  adult  aiding  a toddler,  the  child  must  learn  how  to  balance  on  his  feet  before 
taking  his  first  step,  must  learn  how  to  balance  on  his  first  step  before  stretching  out  for  the  second. 

2.  Decreasing  support  levels 

Scaffolding  emphasizes  the  support  that  the  instructor  can  provide  the  learner  to  reach  beyond  his/her  current 
skill  level.  The  highest  level  of  support  is  the  situation  where  the  instructor  completes  the  task  as  a demonstration 
and  the  learner  takes  no  responsibility  for  the  current  task.  The  lowest  level  of  support  is  the  situation  where  the 
instructor  takes  no  responsibility  for  the  task  and  the  learner  completes  the  task  on  his/her  own.  Between  the 
highest  and  lowest  level  of  support,  the  instructor  shifts  a part  of  the  responsibility  to  the  learner  by  reducing  the 
amount  of  support  one  level  at  a time.  In  the  sense  of  support  withdrawn,  the  instructor  must  recognize  what  kind 
of  support  is  crucial  in  the  learning  and  classify  the  support  into  decreasing  levels.  In  the  example  of  an  adult  aiding 
a toddler,  the  levels  of  support  could  be:  helping  by  holding  two  hands,  helping  by  holding  one  hand,  helping  by 
holding  one  finger,  etc. 

3 . Repetitive  authentic  practice 

The  scaffolding  should  be  performed  in  the  social  context  of  the  practical  work  environment.  In  the 
environment,  the  learner  can  practice  by  taking  part  in  the  expert  behavior  in  a realistic  setting.  Each  practice 
involves  a full  performance  of  the  task.  Besides,  the  underlying  structure  of  the  practice  must  be  repetitive  so  the 
experience  at  one  point  in  task  mastery  could  potentially  be  applied  to  later  activity.  Therefore,  the  instructor  has  to 
set  up  a sequence  of  authentic  practice  involving  the  performance  of  the  same  skills,  in  the  walking  example,  the 
child  actually  experienced  the  practical  task,  walking.  The  practice  of  walking  involves  the  same  component  skills 


303 


O i 

o A o 


o 


and  the  practice  can  be  repeated  in  different  realistic  settings:  walking  on  the  carpet,  walk  on  the  solid  ground,  walk 
on  the  sand,  etc. 

4.  Ongoing  assessment 

In  the  process  of  scaffolding,  the  support  level  should  be  adjusted  based  on  the  learner’s  current  skill  level. 
Therefore,  the  instructor  must  measure  the  learner’s  progress  against  the  global  picture  of  the  task  and  make 
corrections  when  needed.  Usually,  the  completion  of  each  component  skill  is  the  right  time  for  a progress 
assessment.  In  the  walking  example,  the  adult  observes  and  measures  the  child’s  progress  with  each  baby  step.  If 
holding  one  hand  of  the  child  doesn’t  keep  his  balance,  the  adult  gives  him  the  other  hand  right  away.  If  the  child 
walks  well  by  holding  one  hand,  the  adult  will  consider  letting  him  walk  by  holding  only  one  finger. 

Therefore,  an  operational  definition  of  scaffolding  could  be  as  follows.  The  instructor  or  the  more  skillful  peer 
decomposes  the  task  into  hierarchical  sub-tasks,  classifies  the  amount  of  support  in  decreasing  levels,  and  sets  up  a 
repetitive  authentic  practice.  The  practice  begins  with  the  highest  level  of  support  and  the  lowest  level  of  sub-task. 

With  the  completion  of  each  sub-task  in  the  practice,  the  instructor  measures  the  learner’s  performance  and  judges  the 
level  of  support  he/she  should  provide  and  lets  the  learner  perfect  the  component  sub-task  that  he/she  can  manage. 

The  Three-Dimension  Contingent  Scaffolding  Model 

Wood  et  al.  (1976)  and  Day  and  Cordon  (1993)’s  studies  have  suggested  the  “contingent  scaffolded  instruction”, 
emphasizing  the  addition  of  a pattern  of  responses  for  the  withdrawal  of  the  support.  The  pattern  involves  characterizing 
the  instructor’s  intervention  by  the  level  of  abstractness  and  varying  those  levels  systematically  in  response  to  the 
learner’s  performance.  Based  on  the  first  two  elements  of  scaffolding  discussed  above,  there  are  two  types  of  intervention 
that  the  instructor  can  provide  in  a scaffolded  instruction.  The  first  type  involves  the  control  of  the  task  complexity  by 
simplifying  the  task  itself.  The  second  type  involves  the  control  of  the  task  complexity  by  providing  the  additional 
support.  Most  studies  have  focused  on  the  second  type  of  intervention,  the  additional  support.  The  3-D  model  proposed 
in  this  session  adapted  the  contingent  scaffolded  instruction  to  systematically  vary  the  instructor’s  support  in  response  to 
the  learner’s  performance  in  a learning  task  consisting  of  a sequence  of  steps/sub- tasks. 

The  first  two  elements  of  scaffolding  actually  divide  the  learning  task  into  N learning  situations.  Here  N 
indicates  the  number  of  possible  combinations  of  support  level  and  sub-task  level.  For  example,  if  the  target  task 
contains  4 hierarchical  sub-tasks  and  4 levels  of  support,  there  will  be  16  different  possible  learning  situation  that  the 
learner  might  encounter  at  the  time  depending  on  the  learner’s  current  skill  level  (see  the  following  figure). 
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The  support  levels  range  from  level  1,  the  least  independent  and  most  concrete  level  (at  which  the  instructor  assumes  all 
of  the  responsibility  for  the  performance)  to  level  4,  the  most  independent  and  abstract-level  (where  the  learner  has 
assumed  most  of  the  responsibility).  The  sub-tasks  range  from  1 to  4 representing  a sequence  of  acts  which  must  be 
completed  in  this  sequence  in  order  to  complete  the  target  task.  The  process  of  scaffolding  starts  with  the  completion  of 
the  full  task  (from  sub-task  1 to  sub-task  4)  at  support  level  1 and  ends  with  the  completion  of  the  full  task  at  support 
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level  4 through  a sequence  of  repetitive  authentic  practices,  the  third  element  of  scaffolding.  The  whole  process  of 
scaffolding  is  performed  according  to  the  following  rules. 

1 . Each  practice  involves  a full  performance  of  sub-tasks  from  the  first  to  the  last. 

2.  The  first  practice  starts  with  the  highest  level  of  support.  After  that,  each  practice  starts  with  the  level  of  support 
which  is  one  level  lower  than  the  latest  one  used  in  the  previous  practice. 

3.  At  the  current  level  of  support,  the  learner  has  the  chance  to  work  on  the  following  sub-tasks  unless  he/she 
encounters  any  difficulty. 

4.  In  each  sub-task  of  the  practice,  if  the  learner  encounters  any  difficulty,  the  support  level  is  increased  by  one  until 
reaching  the  highest  level. 

The  fourth  element  of  scaffolding,  ongoing  assessment,  is  actually  embedded  in  the  rule  4.  To  see  if  the  learner 
encounters  any  difficulty,  an  ongoing  assessment  must  be  performed  at  the  end  of  each  sub-task. 

This  model  of  scaffolding  describes  the  learning  process  as  a journey  in  the  space  of  the  three  dimensions:  level 
of  sub-task,  level  of  support,  and  number  of  the  repetition  of  the  practice  (see  the  following  figure).  The  first  practice  is 
presented  as  an  example  in  which  the  instructor  takes  full  responsibility  for  the  task  (full  support  is  provided).  The  last 
practice  is  a completely  successful  examination  where  the  learner  takes  the  full  responsibility  of  the  task  (the  least 
support  is  provided).  Any  practice  in  between  either  is  an  unsuccessful  performance  or  a successful  performance  but  with 
some  level  of  support.  Along  the  path  of  the  journey,  each  cubic  block  is  a unit  of  assessment  of  the  learner’s  progress. 
Since  this  model  applies  scaffolding  in  a systematic  way,  it  is  especially  useful  for  the  computer  or  integrated  media 
based  instruction. 


level  of  sub-task 


AN  EXAMPLE  OF  IMPLEMENTATION:  “HYPOTHESIS  TESTING-  THE  Z-TES T” 


Based  on  the  3-D  model,  the  software,  “Hypothesis  Testing—  the  Z-test”,  was  developed.  In  this  section,  we 
describe  the  program  of  “Hypothesis  Testing-  the  Z-test”,  and  explain  how  the  3-D  scaffolding  model  was  implemented 
in  this  program. 


The  Motivation  of  the  Task 

Statistics  is  interesting  and  useful  because  it  is  a mean  of  using  data  to  gain  insight  into  real-world  problems. 
However,  almost  every  statistics  beginner  experiences  difficulties  understanding  the  topic.  Almost  every  statistics 
instructor  admits  that  it  is  not  an  easy  subject  to  teach. 

As  the  continuing  revolution  in  computing  relieves  the  burden  of  calculating  and  graphing,  there  is  an  emerging 
consensus  among  statisticians  that  statistical  education  should  focus  on  data  and  on  statistical  reasoning  rather  than  on 
either  the  presentation  of  as  many  methods  as  possible  or  the  mathematical  theory  of  inference  (Moore  and  McCabe, 

1993).  This  consensus  has  the  following  influence  on  statistical  education. 

1.  Applied  Statistics  should  be  taught  in  the  context  of  real-world  problems. 

Statistics  takes  real  life  patterns  in  real  life  phenomena  and  tests  them  quantitatively.  Bridging  that  gap  between 
reality  and  numbers  is  the  hardest  part.  Not  all  numbers  are  data.  The  number  10.3  acquires  meaning  only  when  we 
are  told  that  it  is  the  birth  weight  of  a child  in  pounds  or  the  percent  of  teenagers  who  are  unemployed.  That  is  the 
context  that  makes  the  numbers  meaningful.  Therefore,  the  examples  and  exercises  should  be  presented  in  the 
context  of  real-world  problems. 

2.  Applied  Statistics  can  be  taught  to  an  extremely  diversified  audience. 

Because  the  statistics  is  useful  for  scientific  investigators  in  many  fields,  the  students  sitting  in  a introduction  course 
of  applied  statistics  might  have  varied  background  in  Agriculture,  Biology,  Chemistry,  Computer  Sciences, 

Education,  Engineering,  Genetics,  Home  Economics,  Industrial  Management,  Mathematics,  Medicine,  Pharmacy, 
Social  Sciences,  Statistics  and  Veterinary  Science.  However,  if  the  student  has  no  fear  of  statistics,  wants  to  learn 
and  possesses  a certain  amount  of  mathematical  maturity,  the  varied  backgrounds  need  not  detract  from  the  success  of 
the  course.  In  fact,  many  times  this  diversity  may  make  students  recognize  similarities  among  problems  from 
different  fields  of  application.  This  gives  them  a broader  scientific  approach  to  statistical  problems  than  providing 
them  only  with  familiar  examples. 

3.  Applied  Statistics  can  be  taught  at  a low  mathematical  level. 

Since  it  is  the  statistical  reasoning  that  should  be  focused  upon  instead  of  the  statistical  computation,  even  students 
with  low  mathematical  level  can  learn  statistics.  Therefore,  the  students  of  applied  statistics  can  range  in 
mathematical  competence  from  Ph.D.  candidates  in  mathematics  to  students  who  had  only  high  school  algebra. 
Actually,  the  learners  need  only  a working  knowledge  of  algebra.  Being  able  to  read  and  use  formulas  would  be 
enough. 

4.  Applied  Statistics  can  be  taught  through  the  applications  of  a set  or  a sequence  of  rules. 

Applied  Statistics  is  actually  the  application  of  a sequence  of  rules  or  principles  which  is  set  up  based  on  certain 
mathematical  proofs  or  the  experience  of  prior  statistical  experts.  The  learners  do  not  have  to  understand  the 
mathematical  proof  behind  each  rule.  However,  it  is  important  for  them  to  understand  the  expert  reasoning  behind 
those  rules.  It  is  not  only  logical  but  also  conceptual.  To  understand  the  statistical  reasoning,  the  learner  has  to 
work  and  communicate  with  the  expert  to  carry  out  the  rules  in  realistic  problems. 

Based  on  the  above  discussions,  we  decided  to  develop  a scaffolded  computer  based  instruction  in  the  context  of 
teaching  basic  applied  statistics.  The  very  basic  test  of  statistic  hypothesis  testing-  the  Z-test  was  selected.  The  task 
was  selected  with  several  objectives  in  mind.  First,  it  had  to  be  challenging  to  the  learner  while  also  proving  sufficiently 
complex  to  ensure  that  his/her  comprehension  and  problem-solving  skill  over  time  could  develop.  Second,  it  had  to  be 
“feature  rich”  in  the  sense  of  possessing  a variety  of  relevant  components.  Third,  its  underlying  structure  had  to  be 
repetitive  so  the  experience  at  one  point  in  task  mastery  could  potentially  be  applied  to  later  activity. 
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This  subject  matter  involves  a sequence  of  problem  solving  steps  and  decision  making.  It  naturally  provides  the 
chance  for  assessing  users'  ongoing  learning  as  well  as  scaffolding  through  program  support.  Therefore,  the  program 
instruction  is  presented  as  a series  of  problems  involving  applying  the  Z-test. 

Implementing  the  3-D  Contingent  Scaffolding  Model  in  “Hypothesis  Testing-  the  Z-Test” 

The  following  table  summarizes  the  four  elements  of  the  scaffolding  model  and  briefly  describes  how  each  is 
implemented. 


Elements  of  Scaffolding  and  “Hypothesis  Testing--  the  Z-Test”  CAI  Design  Features 


Elements  of  Scaffolding 

CAI  Design  Features 

1.  Hierarchical  component  skills 

Four  serial  steps  to  carry  out  the  Z-test 

2.  Decreasing  support  levels 

Four  levels  of  support 

3.  Repetitive  authentic  practice 

Up  to  20  authentic  problem  scenarios 

4.  Ongoing  assessment 

Performance  is  judged  at  the  end  of  each  step 

1 . Hierarchical  component  skills 

The  common  practice  of  hypothesis  testing  includes  of  the  following  steps. 

a.  State  the  hypotheses; 

b.  Find  the  critical  value; 

c.  Compute  the  test  statistic;  and 

d.  Make  the  decision. 

Based  on  the  four  steps,  the  program  decomposes  the  task  into  four  sub-tasks,  allowing  the  learner  concentrate  all 
his/her  effort  on  one  sub-task  at  a time. 

2.  Decreasing  support  levels 

In  considering  the  support  which  can  be  provided  via  integrated-media  and  the  nature  of  the  task,  three  types  of 
support  were  selected  to  be  used  in  the  scaffolded  instruction.  From  concrete  to  abstract,  the  three  types  of  support 
are: 


a.  Visual  Support 

This  is  a graphic  illustration  of  the  problem  situation.  Usually,  the  graph  consists  of  one  or  two  bell-shape 
curves,  representing  the  sampling  distribution(s)  involved  in  the  current  step  situation.  This  kind  of  graphic 
illustration  has  been  used  extensively  in  the  statistics  textbooks  and  statistics  classrooms  when  the  topic  of 
hypothesis  testing  is  First  introduced.  Most  of  the  statistics  teachers  and  learners  believe  this  kind  of  visual 
support  may  facilitate  statistical  reasoning.  In  this  program,  this  kind  of  graphic  illustration  was  used  as  a 
visual  hint  to  the  answer  of  the  current  problem  step. 

b.  Verbal  Support 

This  type  of  support  includes  the  text  instruction  and  leading  questions  or  hints  shown  on  the  screen.  While  the 
answer  of  the  current  step  is  provided  by  the  computer  tutor,  the  verbal  support  is  an  explanation  of  the  answer. 
While  the  answer  should  be  provided  by  the  learner,  the  verbal  support  is  a hint  and  leading  question  of  the 
answer  of  the  current  problem  step.  For  example,  ‘The  first  step  of  the  hypothesis  testing  is  to  state  the  Null 
and  Alternative  Hypotheses.  What  hypotheses  can  you  state  for  this  problem?” 
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c.  Symbolic  Support 


This  type  of  support  could  be  a specific  Greek/English  letter  with  a pre-defined  meaning  or  a mathematical 
symbol  for  operations.  Having  the  statistical  reasoning,  the  practice  of  statistics  can  always  be  represented  by 
formula,  symbols,  and  numbers.  The  four  steps  of  the  hypothesis  testing  can  also  be  represented  in  symbols. 

In  practice,  the  symbolic  support  took  the  form  of  a symbolic  prompt,  requesting  the  learner  to  provide  the 
answer  of  the  current  problem  step.  For  example,  “Za  prompts  the  learner  to  find  the  critical  value  of  the  test 

statistic  Z using  the  given  level  of  significance  a. 

The  program  was  developed  using  Visual  BASIC  3.0  for  the  Windows  environment.  The  actual  environment  of  the 
lesson  is  displayed  in  the  following  figure.  It  explains  how  the  visual,  verbal,  and  symbolic  support  show  on  the 
program  screen. 


— | Hypothesis  Testiriq  M A 

New 

Problem  Area 

Next 

Instruction  Area 

iijlHi 

ill 

llilliallilll 

:::::::::  ::t: 

Help] 
Calc  J 

fSTEP  1 LEUELJ 

Figure  Area 

Answer  Area 

: 

End 

Hit  lim!iiiiii!l!i  !!!i!iiimiim  !ii!iiiiiiii!ii!  I!il!!!I!!II!Il!  !!!i!i!i!!i!!!!!  H!i 

The  ertviiomnent  of  the  lesson. 


1 . Problem  Area 

The  Problem  Area  shows  the  current  problem  scenario  that  the  student  is  working  on. 

2.  Instruction  Area 

The  Instruction  Area  provides  the  instruction  or  hints  to  solve  the  current  step  of  the  problem.  It  counts  as  the 
verbal  support. 

3.  Figure  Area 

The  Figure  Area  illustrates  the  current  step  situation  of  the  problem  in  a graphic  form  based  on  the  learner’s 
selection  of  the  answers.  That  is  where  the  visual  support  takes  place. 

4 . Answer  Area 

The  Answer  Area  provides  the  answer  or  requests  the  learner  to  provide  answer  for  the  current  step  after  the 
symbolic  support. 
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Levels  of  Program  Support 


Based  on  the  three  types  of  support  described  above,  the  levels  of  support  can  be  classified  and  ranked  by  the 
amount  of  support  (how  many  types  of  support)  provided  at  the  time.  These  are  described  below.  From  level  1 to 
level  4,  the  next  most  concrete  type  of  support  is  withdrawn  from  the  previous  level  of  support. 


Level  1 : Full  Support 


The  instruction  demonstrates  the  steps  to  solve  the  problem  in  detail  with  visual,  verbal,  and  symbolic 
information.  The  following  is  an  example  of  the  full  support  screen. 


Hypothesis  Testing 


New 


Next 


1 . Rat*  that  are  raised  in  a laboratory  environment  have  a mean  life  span  of  around  24  month*.  A 
sample  or  3B  rat*  reared  to  adultnood  In  a germ-tree  environment  had  lire  span*  with  a mean  or  27. 
Assuming  a population  standard  deviation  of  6 months,  does  this  type  of  rearing  increase  the  life 
span  of  the  laboratory  rats  to  a point  that  is  enough  to  reject  the  null  hypothesis  at  0.05  level  of 
significance?  (McGhee.  1985  . p331.  Example  9.13) 


From  the  problem  statements,  we  expect  the  mean  p of  the 
population  from  which  the  sample  was  drawn  will  be  greater  than  24 . 
Therefore,  the  null  hypothesis  Ho  and  the  alternative  hypothesis  Hi. 


rSTEP_1  LEVEL.! 


End 


HO 

/> 

HI 

cv 

24 

written  in  symbols,  are 
HO:  u = 24 
HI:  u > 24. 

A one-tailed  test  to  the 
right  is  required. 


Level  2:  Visual,  Verbal,  and  Symbolic  Support 

Instead  of  providing  the  answer  with  detailed  explanation,  the  instruction  only  provide  the  visual  and 
verbal  hints  to  the  current  problem  step  and  requests  the  learner  to  give  the  answer  after  the  symbolic 
prompts.  An  example  of  the  screen  with  level  2 support  is  provided  in  the  following  figure. 
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— 

Hypothesis  Testing 

New 

2 In  several  studies  it  has  been  reported  that  the  natural  age  at  menopause  of  non  smoking  women 
is  around  30  years:  In  a study  entitled  Cigarette  Smoking  and  Natural  Health,  a sample  of  43  heavy 
smokers  had  a mean  age  at  menopause  of  48  years.  Assuming  a population  standard  deviation  of 
5.6  years,  test  the  hypothesis  that  cigarette  smoking  is  associated  with  early  onset  of  menopause 
with  0.05  level  of  significance.  (McGhee,  1985,  p332.  Example  9.14) 

Next! 

The  first  step  of  the  hypothesis  testing  is  to  state  the  Null  and 
Alternative  Hypotheses.  What  hypotheses  can  you  state  for  this 

problem?  C|jck  one  ^ the  buttons  to  decide  the  direction  of  the  alternative  hypothesis. 

STEPJI  LEVEL.2 

/V\ 

> 

HO:  u = 50 

yjvV 

< 

HI:  u05O 

The  Figure  Area  in  the  example  is  an  initial  form  of  the  graphic  illustration  of  the  current  step,  Step  1 
Based  on  the  learner’s  selection  of  the  answers,  more  information  will  be  added  to  the  graph.  The 
following  figure  shows  the  result  if  the  “>”  is  selected  in  the  answer  area. 
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On  the  other  hand,  if  the  “<”  is  selected,  the  result  will  be: 


Level  3:  Verbal  and  Symbolic  Support 


The  information  in  the  figure  area  disappears.  Only  the  verbal  hint  is  provided,  telling  the  learner  some 
specific  information  to  look  for  and  asking  him/her  to  provide  answers  after  the  symbolic  prompts.  The 
following  figure  is  an  example  screen  with  this  level  of  support. 


— 

Hypothesis  Testing 

Ntt.w 

3.  A survey  of  faculty  workloads  at  a large  university  in  1 979  showed  an  average  of  44.3  hours  per 
week  or  Instructional  and  academic-related  work.  Recently  the  university  switched  to  the  quarter 
system.  Assuming  a population  standard  deviation  of  6 hours,  test  the  hypothesis  that  the  workload 
is  unchanged  and  did  not  increase  as  predicted  if  a random  sample  of  38  faculty  members  showed 
workloads  with  a mean  of  45.7  hours  per  week.  Use  0.05  as  the  level  of  significance.  (McGhee, 
1985.  p333.  Exercise  Set  9.3.  PartA.  #5] 

&CKt 

The  first  step  of  the  hypothesis  testing  is  to  state  the  Null  and 
Alternative  Hypotheses.  What  hypotheses  can  you  state  for  this 

problem?  click  one  „f  button*  to  decide  the  direction  of  the  alternative  hypothesis. 

rSTEPJI  LEVEL_3 


HO:  u = 44.3 
HI:  uQ 44.3 


The  visual  hints  of  the  current  step  have  been  taken  away. 

Level  4:  Symbolic  Support 

The  instruction  takes  away  the  next  most  concrete  support  from  the  screen.  Therefore,  only  the  symbolic 
prompts  will  be  provided,  requiring  the  learner  to  give  answers.  The  following  screen  shows  the  problem 
step  1 with  least  level  of  the  support. 
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— Hypothesis  Testing 

New 

4.  Suppose  the  I.Q.s  of  graduating  high  school  seniors  at  a metropolitan  area  have  a standard 
deviation  of  24.  The  metropolitan  area  selects  54  students  at  random  and  wit.  wte  them  to 
demonstrate  (hopefully)  that  students  from  this  area  have  I.Q.s  exceeding  the  national  average  of 
105.  The  sample  of  students  showed  I.Q.s  with  a mean  of  111.  What  is  the  conclusion  if  the  level 
of  significance  is  0.01.  (Mc6hee.  1985,  p333.  Exercise  Set  9.3,  PartA,  *G) 

Next 

Click  one  of  the  buttons  to  decide  the  direction  of  the  alternative  hypothesis. 

STEPJI  LEVEL_4 

> HO:  u = 105 

rn  HI:  u[]l05 

15 

Not  only  has  the  visual  support  been  taken  away,  the  verbal  hints  are  no  longer  shown  on  the  screen 
either. 

The  combination  of  the  4 component  steps  of  the  task  and  the  4 levels  of  the  support  makes  16  possible  learning 
situations: 


LEVEL  1: 
Full  Support 

LEVEL  2: 
Visual,  Verbal, 
& Symbolic 
Support 

LEVEL  3: 
Verbal  & 
Symbolic 
Support 

LEVEL  4: 
Symbolic 
Support 

STEP  1 : 
State  the 
Hypotheses 

STEP  2: 
Find  the  Critical 
Value 

STEP  3: 
Compute  the 
Test  Statistic 

STEP  4: 
Make  the 
Decision 

Based  on  the  16  learning  situations,  16  templates  of  the  information  screens  were  created  in  the  program. 
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3 . Repetitive  authentic  practice 

The  problems  were  adapted  from  examples  in  statistics  text  books,  covering  the  real  problems  in  many  disciplines: 
agriculture,  biology,  engineering,  pharmacy,  business,  and  the  social  sciences. 

4.  Ongoing  assessment 

At  the  end  of  each  step,  the  learner’s  answer  is  judged  for  correctness.  This  judgment  decides  the  next  learning 
situation  the  learner  will  encounter. 

Recall  the  navigation  rules  of  the  3-D  learning  journey  described  in  the  previous  section: 

1 . Each  practice  involves  a full  performance  of  steps  t to  4; 

2.  The  first  practice  starts  with  the  highest  level  of  support-  Level  1.  After  that,  each  practice  starts  with  the  level  of 
support  which  is  one  level  lower  than  the  latest  one  used  in  the  previous  practice; 

3.  At  the  current  level  of  support,  the  learner  has  the  chance  to  work  on  the  following  sub-tasks  unless  he/she 
encounters  any  difficulty. 

4.  In  each  step  of  the  practice,  if  the  learner  encounters  any  difficulty,  the  support  level  is  increased  by  one  until 
reaching  the  highest  level-  Level  1. 

If  we  project  all  the  possible  3 dimensional  learning  journeys  on  to  the  2 dimensional  table  of  the  16  learning  situations, 
the  possible  learning  paths  are: 


LEVEL  1: 
Full  Support 
START 

LEVEL  2: 
Visual,  Verbal, 
& Symbolic 
Support 

LEVEL  3: 
Verbal  & 
Symbolic 
Support 

LEVEL  4: 
Symbolic 
Support 

STEP  1: 
State  the 
Hypotheses 

> 

tit  

*xr  _ 

^ 

A 

R 

a 

iT 

A 

LR 

STEP  2: 

Find  the 
Critical  Value 

> 

L 

/j 

t« 

/J\ 

L 

1 

/ w 

R 

/ w 

STEP  3: 
Compute  the 
Test  Statistic 

> 

L 1 

uJ 

> 

f 

R 

V 

W 

R/ 

W 

W 

STEP  4: 

Make  the 
Decision 

> 

J 

& 

> 

J 



vv  ^ 

W ^ 

R : if  the  right  answer  is  provided 
W : if  a wrong  answer  is  provided 


-END 


We  believe  this  model  provides  a systematic  way  to  integrate  the  method  of  scaffolding  and  the  integrated-media 
based  instruction.  Because  this  model  not  only  bridges  the  learner’s  skill  levels  but  also  effectively  withdraws  support, 
the  learner  can  finally  become  an  active  and  independent  learner.  Therefore,  this  specific  scaffolded  computer-based 
instruction,  “Hypothesis  Testing-  the  Z test’*,  was  expected  to  be  an  effective  and  powerful  instructional  program, 
inheriting  the  advantages  and  minimizing  the  limitations  of  scaffolding.  In  this  study,  this  scaffolded  instruction  was 
evaluated  in  terms  of  comprehension,  knowledge  maintenance,  and  knowledge  transfer  by  comparing  it  to  the  full-support 
instruction  (in  which  the  full  support  is  available  all  the  time),  and  the  least-support  instruction  (in  which  only  the 
symbolic  support  is  provided,  prompting  for  the  answers). 
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EVALUATION  OF  'HYPOTHESIS  TESTING-  THE  Z-TEST' 


"Hypothesis  Testing-  the  Z-Test"  has  been  evaluated  to  address  three  major  questions  and  corresponding 
hypotheses: 

• Would  the  availability  of  full  support  in  the  computer-assisted  learning  hamper  learner’s  independence?  Our 
hypothesis  was  that  the  students  in  the  full  support  condition  would  not  perform  better  than  the  students  in  the  least 
support  condition  (the  control  condition)  in  the  maintenance  and  transfer  posttests,  where  the  support  was  no  longer 
available. 

• Would  scaffolded  instruction  enhance  knowledge  maintenance  in  a computer-assisted  learning  environment?  Our 
hypothesis  v as  that  students  in  the  scaffolded  condition  would  perform  better  in  the  maintenance  posttest  than 
students  in  the  other  two  conditions. 

• Would  scaffolded  instruction  promote  knowledge  transfer  in  a computer-assisted  learning  environment?  Our 
hypothesis  was  that  students  in  the  scaffolded  condition  would  perform  better  in  the  transfer  posttest  than  students  in 
the  other  two  conditions. 

A secondary  interest  was  about  the  subject  matter  itself. 

• Could  the  subject  of  “Hypothesis  Testing”  be  taught  at  low  mathematical  level?  Our  hypothesis  was  that  students 
with  low  mathematical  skills  would  also  benefit  from  this  scaffolded  instruction. 

The  Experiment 

Research  Design 

Based  on  the  research  questions  and  hypothesis  described  above,  a baseline-posttest  control  group  design  was 
implemented  using  the  computer  program,  "Hypothesis  Testing—  the  Z-test  . All  the  hypotheses  were  analyzed  under  a 
regression  model. 

Regressor  Variables 

There  were  two  types  of  regressor  variables  considered  in  this  study. 

1.  Baseline  Variables 

The  learner's  mathematical  and  statistical  background  was  measured  from  six  different  aspects.  They  were: 
courses  taken  in  Mathematics  (MATH_C),  courses  taken  in  Statistics  (STATIC),  self-rating  mathematical  skills 
(MATH_S),  self-rating  statistical  skills  (STATES),  mathematics  preference  (MATHJP),  and  statistics  preference 
(STATJP). 

Besides,  the  pretest  score  was  also  collected  as  a baseline  measure  of  their  prior  knowledge  about  the  topic  to  be 

taught. 

2.  Treatment  Variable—  the  support  condition 

To  test  the  effectiveness  of  the  scaffolded  instruction,  the  full  support  and  the  least  support  versions  of 
“Hypothesis  Testing-  the  Z-Test”  were  also  created.  The  three  different  instructional  conditions  are  described  below. 

Scaffolded  Condition  (adjusts  level  of  support  among  1, 2,  3.  and  4 in  an  orderly  fashion  depending  on  the  learner’s 
current  ability) 

Based  on  the  four  levels  of  support,  the  program  started  from  the  level  of  full  support.  The  amount  of  support 
was  reduced  by  one  level  in  the  next  problem  as  the  user  demonstrated  the  ability  to  solve  the  present  problem  at  the 
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current  level  of  support.  Once  the  user  encountered  difficulty  at  any  step  of  the  problem-solving  process,  the  amoui.*  A 
the  support  would  be  increased  by  one  level  for  the  current  problem  step  immediately.  The  teaching  process  proceeded 
until  the  user  could  solve  two  consecutive  problems  with  the  lowest  level  of  support. 

Full  Support  Condition  (swing  between  levels  of  support  1 and  4) 

This  condition  represents  the  support  condition  which  has  been  used  in  many  computer-assisted  learning 
environments.  The  program  first  demonstrated  the  complete  problem  solving  procedure  in  a full-support  example.  Then 
it  gave  another  problem  with  the  least  amount  of  support  (i.e.  symbolic  support)  as  a test.  Full  support  was  a screen 
option  which  can  provide  immediate  help  if  the  user  encountered  any  difficulty  in  the  problem-solving  process  (see  the 
following  figure).  If  the  user  did  not  ask  for  full  support  but  made  any  mistake  in  any  step  of  the  problem  solving 
procedure,  the  full  support  would  be  provided  starting  from  the  current  problem  step.  The  current  problem  would  then  be 
solved  as  an  example  and  the  next  problem  would  be  presented  with  the  least  level  of  support  as  a test  again.  That  is,  the 
program  support  swung  between  the  levels  of  full  support  and  least  support.  The  teaching  process  proceeded  until  the 
user  could  solve  two  consecutive  problems  with  the  least  amount  of  support. 


Hypothesis  Testing 

Uvw 

2.  In  several  studies  it  has  been  reported  that  the  natural  age  at  menopause  of  non  smoking  women 
Is  around  30  years,  in  a study  entitled  cigarette  Smoking  and  Natural  Health,  a sample  of  43  heavy 
smokers  had  a mean  ape  at  menopause  of  48  years.  Assuming  a population  standard  deviation  of 
5.8  years,  test  the  hrpothesis  that  cigarette  smoking  is  associated  with  early  onset  of  menopause 
with  0.05  level  of  significance.  (McGhee,  1985,  p332.  Example  9.14) 

Next 

Click  one  of  the  buttons  to  decide  the  direction  of  the  alternative  hypothesis. 

rSTEPJ  LEVELS - 


Help 


i£ndj 


HO:  u = 50 
HI:  u[] 50 


Least  Support  Cotidition  (use  level  of  support  1 once,  then  level  of  support  4 for  the  rest) 

This  condition  of  program  support  represents  the  traditional  teaching  method  which  has  been  used  in  many 
statistic  textbooks.  It  served  as  a control  condition.  As  in  the  other  two  treatment  groups,  the  program  first  explained 
the  four  steps  of  hypothesis  testing  procedure  in  a sample  Z-test  problem  thoroughly  (i.e.  with  full  support).  However, 
in  the  following  problems,  only  the  least  amount  of  support  (i.e.  symbolic  support)  was  provided.  The  user’s  response 
in  each  step  was  judged  for  correctness  at  the  end  of  each  problem.  If  any  mistake  occurred  in  any  step  of  the  solving 
procedure,  the  correct  answers  would  be  provided  without  explanation  (see  the  following  figure).  The  process  proceeded 
until  the  user  solved  two  consecutive  problems. 
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Nett 


The  discharge  temperature  of  coolant  water  from  a nuclear  reactor  is  stipulated  to  have  a mean 
of  45.  An  environmental  group  believes  it  to  be  higher  and  on  3B  different  occasions  obtains 
temperature  readings  that  have  a mean  of  46.8.  Assuming  a population  standard  deviation  of  3.G, 
cany  out  the  test.  Use  0.025  as  the  level  of  significance.  (McGhee,  1385,  p333.  Exercise  Set  9.3, 
PartA,  87) 


Next 


Z = 3 

Za=  164 

Click  one  of  the  b 
rSTEP_4  LEVEL 


End 


Please  check  your  answers: 


The  answers  should  be: 

HO:  u - 45 
HI:  u > 45 
Za~  1.96 
Z - 2.99 

Decision:  reject  HO 


i m i 


ect 


HO 


Accept 


All  conditions  of  the  instructional  program  accessed  the  problems  from  a data  file  which  contained  20  problems 
involving  Z-test  about  one-tailed  single  sample  mean. 


Two  response  variables  were  involved  in  this  study.  They  were  students’  scores  on  the  maintenance  posttest  and  transfer 
posttest. 

Subjects 

Subjects  were  72  students  enrolled  in  an  introductory  course  of  educational  psychology  at  Purdue  university  in 
the  Fall  semester  of  1995.  The  majority  of  the  students  were  juniors  and  sopromores  in  educational  majors. 

Before  their  participation,  the  concepts  of  the  normal  distribution,  standard  deviation,  and  the  Z-score  had  been 
covered  in  the  first  6 weeks  of  the  course.  Therefore,  the  materials  of  instructional  program  were  relevant  and  appropriate 
to  the  students. 

The  participation  of  the  study  was  counted  as  two-hour  research  credits.  Students  were  invited  on  a volunteer 
basis.  Subjects  were  then  randomly  assigned  into  the  three  condition  groups. 

Instruments 

All  the  tests  were  administered  in  paper-and-pencil  format  with  a calculator  and  a copy  of  the  Z-score  table 

provided. 

Pretest  and  Maintenance  Posttest 

Each  test  consisted  of  three  problems  involving  a one-tailed  Z-test  of  single  sample  means.  Each  problem 
consisted  of  four  steps  and  each  step  involves  at  least  three  different  subconcepts.  Therefore,  a total  score  of  48  was 
possible. 

Transfer  Posttest 

It  contained  one  one-tailed  and  two  two-tailed  Z-test  problems  of  single  sample  means.  Each  problem  consisted 
of  four  steps  and  each  step  involves  at  least  three  different  subconcepts.  Therefore,  a total  score  of  48  was  possible. 

All  the  test  probler  «s  were  adapted  from  statistic  textbooks  and  previewed  by  two  experts  in  statistics. 
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Procedure 


The  experiment  was  conducted  in  the  seventh  to  tenth  weeks  of  the  Fall  semester  of  1995.  It  consisted  of  two 
sessions  with  7 to  10  days  in  between.  For  better  control  of  the  experiment,  the  first  session  was  conducted  by  one  of 
the  researchers,  one  student  at  a time. 

Session  One 

The  participant  first  completed  a participant  survey  at  the  beginning  of  the  first  session  of  the  experiment.  Then 
the  participant  was  informed  of  the  session  agenda  and  the  expected  time  spent  on  each  activity.  After  that,  the  researcher 
checked  the  participant’s  mouse  skill  and  demonstrated  the  skill  if  necessary.  Before  working  on  the  computer,  the 
participant  was  provided  with  a simple,  hand-held  calculator,  a Z-score  table,  and  a couple  of  worksheets  for  notetaking 
and  calculation.  The  participant  could  keep  and  us?  these  tools  for  the  rest  of  the  session.  The  participant  then  started  to 
work  on  the  pre-instruction  in  front  of  the  computer  foi  about  20  minutes.  The  pre-instruction  covered  all  the 
prerequisite  skills  needed  for  the  major  instruction.  Then,  for  10  minutes,  the  participant  had  a chance  to  self-evaluate  the 
key  concepts  learned  in  the  pre-instruction  by  working  on  the  paper-and-pencil  prerequisite  test.  A key  with  detailed 
explanation  was  provided.  After  the  prerequisite  test,  the  participant  was  given  a three-problem  pretest  to  work  on.  The 
participant  was  told  at  the  beginning  that  he/she  was  not  expected  to  know  the  answers  to  these  problems.  If  having 
spent  more  than  three  minutes  on  one  problem  but  was  unable  to  proceed,  he/she  could  quit  the  pretest  there.  The 
expected  time  for  the  pretest  was  10  minutes.  After  completing  the  pretest,  the  participant  went  back  to  the  computer 
screen  and  work  on  the  major  instruction  for  45  minutes.  The  instruction  started  with  a short  introduction  of  the  lesson 
environment.  After  that,  the  program  demonstrated  a step  by  step  procedure  to  carry  out  the  Z-test  in  a problem  example. 
Then  the  participant  encountered  a series  of  similar  problems  requiring  him/her  to  provide  the  answers  step  by  step. 

Along  with  these  problems,  one  of  the  program  support  conditions  (scaffolded,  full  support,  or  least  support)  was 
applied,  depending  on  which  condition  group  the  participant  had  been  assigned  into.  The  program  could  not  be 
terminated  until  the  participant  solved  two  consecutive  problems  with  the  least  amount  of  support.  However,  the 
researcher  would  ask  the  participant  to  stop  if  he/she  had  spent  more  than  45  minutes  on  the  major  instruction  and  not 
finished.  During  the  instruction,  the  participant  could  either  use  the  computer  and  the  Z-score  table  at  hand  or  the  ones 
on  the  computer  screen.  The  participant’s  learning  path  and  responses  were  recorded  in  an  output  file.  The  number  of 
problems  the  participant  went  through  either  to  mastery  or  by  reaching  the  time  limit  was  noted. 

Session  Two 

This  session  was  held  about  7 to  10  days  after  the  instruction  session.  The  subject  first  was  asked  to  complete 
the  three-problem  maintenance  posttest  for  20  minutes.  After  the  test,  a one-page  instruction  on  two-tailed  Z-test  was 
provided.  After  10  minutes,  the  instruction  sheet  was  taken  away  and  the  three-problem  near  transfer  posttest  was 
administered.  Again,  the  subjects  could  work  on  the  test  for  20  minutes.  Upon  completion  of  the  test,  the  solutions  to 
all  the  test  problems  were  provided  to  the  subject. 

Data  Analysis  and  Results 


One-way  ANOVA  was  first  conducted  on  pretest  to  establish  pre-study  equivalence.  Then,  regression  analysis 
was  applied  to  both  response  variables  (Maintenance  Posttest  and  Transfer  Posttest)  with  the  eight  regressor  variables 
(Support  Condition,  Pretest,  MATH_C,  STAT_C,  MATHIS,  STAT_S,  MATH_P,  and  STAT_P). 

Pretest 

The  following  table  shows  the -results  of  the  test  for  the  group  difference  on  pretest. 


Group 

Sample 

Size 

Mean 

Std  Dev 

F Value 

Pr  > F 

Full  Support 

24 

4.25 

5,28 

0.59 

0.5554 

Scaffolded 

26 

4.26 

3.68 

Least  Support 

22 

5.54 

4.81 

kJ  i f J 
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All  the  three  groups  got  low  average  scores  on  pretest.  No  significant  difference  was  found  among  groups. 

Maintenance  Posttest 

The  sample  size,  mean,  standard  deviation,  and  the  least  square  mean  on  maintenance  posttest  for  each  treatment 
group  (support  condition)  are  listed  in  the  following  table. 


Support 

Condition 

Sample  Size 

Mean 

Standard 

Deviation 

Least 

Square 

Mean 

Full  Support 

24 

22.20 

13.23 

21.43 

Scaffolded 

26 

26.92 

11.98 

28.19 

Least  Support 

22 

19.70 

12.29 

19.07 

The  treatment  effect  on  maintenance  posttest  was  examined  in  the  regression  analysis  along  with  other  regressor 
variables.  The  following  table  presents  a summary  of  the  significance  data  for  regression  analysis  on  the  maintenance 
posttest. 


Response  Variable:  Maintenance  Posttest 


Source 

F Value 

Pr  > F 

Model  (R-Square  = 0.388188) 

4.37 

0.0002 

Support  Condition 

2.89 

0.0227 

Pretest 

6.53 

0.0130 

The  data  from  this  analysis  showed  significant  effects  for  the  treatment  variable-  the  support  condition  and  the  pretest. 
The  effects  of  the  rest  of  the  regressor  variables  were  not  significant  at  0.05  level  of  significance.  Pairwise  t-test s were 
conducted  on  the  support  condition  as  a post  hoc  analysis.  The  following  table  presents  the  results. 


Support 

Condition 

Full 

Support 

Scaffolded 

Least 

Support 

Full  Support 

- 

t = -2.05 
p = 0.0442 

t = 0.73 
p = 0.4648 

Scaffolded 

t = 2.05 
p = 0.0442 

- 

t = 2.75 
p = 0.0078 

Least  Support 

t = -0.73 
p = 0.4648 

t = -2.75 
p = 0.0078 

- 

The  least  square  mean  score  of  the  scaffolded  group  was  significant  higher  than  the  other  two  groups  at  0.05  level  of 
significance.  No  significant  difference  were  found  between  the  full  support  group  and  the  least  support  group. 

Transfer  Posttest 

The  sample  size,  mean,  standard  deviation,  and  the  least  square  mean  on  transfer  posttest  for  each  treatment 
group  (support  condition)  are  listed  in  the  following  table. 


Support 

Condition 

Sample  Size 

Mean 

Standard 

Deviation 

Least  Square 
Mean 

Full  Support 

24 

25.39 

15.54 

26.50 

Scaffolded 

26 

30.98 

10.15 

29.58 

Least  Support 

22 

27.50 

13.22 

27.94 

The  treatment  effect  on  transfer  posttest  was  examined  in  the  regression  analysis  along  with  other  regressor 
variables.  The  following  table  shows  the  significant  results  of  regression  analysis  on  the  transfer  posttest. 
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Response  Variable:  Transfer  Posttest 


Source 

F Value 

Pr  > F 

Model  (R-Square  = 0.376209) 

4.15 

0.0003 

Pretest 

6.36 

0.0143  i 

STAT.C 

4.76 

0.0329 

MATHJS 

6.97 

0.0105 

The  preceding  data  suggested  that  the  transfer  posttest  score  was  significantly  influenced  by  the  pretest  score,  the  statistics 
courses  taken  before,  and  the  learner’s  self-rating  mathematical  skill  The  effects  of  the  rest  of  the  regressor  variables 
were  not  significant  at  0.05  level  of  significance,  including  the  treatment  effect  (the  support  condition). 

Discussion 

Maintenance  Posttest 


The  Support  Condition 

The  results  support  our  first  hypothesis.  The  students  in  the  full  support  condition  did  not  perform  significantly 
better  than  the  control  group  (i.e.  the  least  support  group).  The  availability  of  full  support  in  the  program,  “Hypothesis 
Test—  the  Z-Test”,  did  hamper  learner’s  independence.  On  the  other  hand,  the  results  indicate  that  the  scaffolded 
instruction  was  quite  successful  in  this  study.  The  scaffolded  group  got  significantly  higher  average  score  than  the  other 
two  groups.  Our  second  hypothesis  was  also  verified. 

This  finding  is  consistent  to  the  suggestion  proposed  by  the  Cognition  and  Technology  Group  at  Vanderbilt 
University  (1993).  To  avoid  the  learner  over  relying  on  the  support  provided  in  the  integrated  media,  its  designs  should 
function  to  provide  scaffolding  rather  than  always  provide  full  support. 

Pretest  Score 

It  is  not  surprising  that  the  students’  performance  on  maintenance  posttest  was  influenced  by  their  prior 
knowledge  on  the  topic.  Actually,  the  results  also  shows  that  most  of  the  students  did  not  h e any  prior  knowledge  on 
the  target  learning  task  (applying  the  Z-test)  except  the  very  basic  statistic  concepts  taught  in  the  introductory  course 
(such  as  normal  distribution  and  standard  deviation).  However,  the  understanding  of  these  very  basic  concepts  (which  was 
reflected  on  the  pretest  score)  might  improve  the  learner’s  statistical  reasoning  on  the  learning  task  (which  was  reflected 
on  the  posttest  scores).  Basically,  the  background  of  this  subject  group  was  very  homogeneous.  No  significant 
differences  were  found  among  the  three  treatment  groups. 

Other  Baseline  Measures  on  Learners’  Mathematical  and  Statistical  Background 

The  results  show  no  significant  effect  for  any  of  these  baseline  variables.  The  finding  on  learners’  mathematical 
background  supports  our  last  hypothesis.  Since  the  maintenance  posttest  score  was  not  affected  by  the  learner’s 
mathematical  background,  students  with  low  mathematical  skills  would  also  benefit  from  this  scaffolded  instruction. 

Only  4 to  6 students  in  each  group  had  ever  taken  Statistics  course  before.  The  finding  of  no.  significant  effects  of 
learners’  statistical  background  also  shows  that  the  content  can  be  taught  to  statistical  beginners. 

Transfer  Posttest 


The  Support  Condition 

The  findings  failed  to  support  our  third  hypothesis.  The  transfer  posttest  score  was  not  significantly  influenced 
by  the  support  condition.  There  are  four  possible  explanations: 

1 . The  two-tailed  tests  on  the  transfer  instrument  actually  involved  the  same  four  steps  as  the  one-tailed  tests.  The  one 
page  example  of  a two-tailed  case  gave  the  learner  a chance  to  review  and  recall  the  four  steps. 
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2.  Instead  of  showing  one  step  at  a time  on  the  screen,  the  one  page  example  presented  the  four  steps  all  together  on 
the  same  page.  It  have  given  the  learner  a whole  view  of  the  statistical  reasoning. 

3.  The  two-tailed  tests  did  not  involve  deciding  the  direction  of  the  alternative  hypothesis.  However,  that’s  the  most 
crucial  concept  in  the  one-tailed  tests.  Many  students  expressed  that  the  two-tailed  cases  were  easier  than  the  one- 
tailed  cases. 

4.  The  treatment  condition  may  promote  immediate  learning  but  not  transfer. 

Therefore,  most  of  the  learners  got  a higher  score  on  the  transfer  posttest.  Although  the  group  difference  on  transfer 
posttest  did  not  reach  the  significant  level,  the  scaffolded  group  still  got  a higher  average  score  and  a smaller  standard 
deviation  than  the  other  two  groups.  One  interesting  finding  was  that  the  least  support  group  got  a higher  mean  score 
and  a smaller  standard  deviation  than  the  full  support  group.  This  might  be  explained  by  the  least  support  group  having 
a higher  pretest  score  or  the  least  support  group  actually  needed  to  spent  more  mental  effort  to  carry  out  the  test  on  their 
own. 

The  Pretest 

The  effect  of  the  pretest  also  was  apparent  on  the  transfer  posttest.  This  result  was  expected. 

Other  Baseline  Measures  on  Learners’  Mathematical  and  Statistical  Background 

Only  the  courses  taken  in  Statistics  (weighted  by  the  course  level)  and  the  self-rating  mathematical  skills  (on  a 
scale  of  10)  showed  significant  effects  for  the  transfer  posttest  score.  This  result  suggests  that  students  having  more 
statistical  background,  feeling  more  confident  about  their  mathematical  skills  transferred  the  old  skill  to  the  new  learning 
situation  better. 


CONCLUSIONS 

In  this  paper,  the  3-D  contingent  scaffolding  model  has  been  discussed,  implemented,  and  evaluated  on  a 
computer-based  instruction,  “Hypothesis  Testing-  the  Z-Test”  in  order  to  establish  baseline  data  for  integrated-media- 
based  instruction  and  hypermedia  based  learning  environment.  The  results  indicate  that  this  model  can  be  successful  in 
promoting  learning.  This  model  provides  a systematic  way  to  link  the  concept  of  scaffolding  to  the  integrated  media 
design  features  using  both  support  building  and  fading  techniques.  It  could  be  adapted  to  other  integrated  media  teaming 
environments  and  for  other  content  and  other  audiences. 
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The  Simplifying  Conditions  Method  (SCM)  is  a relatively  new  set  of  guidelines  for  task  analysis  and  sequencing  of 
instructional  content  under  the  Elaboration  Theory  (ET)  (Reigeluth  & Rogers,  1980;  Reigeluth  & Stein,  1983).  During  the 
last  20  years,  even  though  ET  has  been  one  of  the  most  well-received  and  extensively  referred  to  method  used  by  practitioners 
and  researchers  (Wilson  & Cole,  1992),  SCM  has  received  relatively  less  attention  compared  to  its  potential  strength  as  a tool 
for  task  analysis  and  sequencing. 

Initially,  SCM  was  developed  for  the  procedural  task  (focusing  on  "how")  by  Reigeluth  and  Rogers  (1980);  it  has 
been  elaborated  and  proven  that  SCM  also  works  well  for  the  transfer  task  (focusing  on  "why")  (Reigeluth  & Kim,  1991). 
Since  1991,  we  have  invested  a lot  of  effort  into  improving  and  extending  the  scope  of  SCM  while  incorporating  knowledge 
from  the  current  advances  of  cognitive  psychology  and  constructivism  (Reigeluth,  1992;  Wilson  & Cole,  1992)  and  from  the 
several  empirical  field  applications  and  testings  (Kim,  1994;  Reigeluth  & Kim,  1991,  1992,  1993).  Since  our  findings 
provided  meaningful  information,  giving  us  a relatively  high  level  of  confidence  in  SCM,  we  conducted  formative  research  on 
SCM  to  finalize  and  synthesize  our  efforts. 

This  article  introduces  the  fundamentals  of  SCM  and  shares  the  findings  from  our  formative  research  on  SCM. 

Fundamentals  of  the  Simplifying  Conditions  Method  (SCM) 

SCM  was  developed  to  add  more  detail  to  the  Elaboration  Theory,  which  resulted  from  Reigeluth's  intensive  work  to 
integrate  the  knowledge  base  of  instructional  and  learning  theories  into  a set  of  prescriptions  at  the  macro-level  (Reigeluth  & 
Stein,  1983).  Consequently,  the  most  important  ideas  of  ET  are  based  upon  Gagn6's  (1977)  hierarchical  task  analysis  and 
sequence,  Bruner's  (1960)  spiral  curriculum,  Ausubel's  (1963)  progressive  differentiation,  and  Merrill's  (1978)  and  Scandura's 
(1973,  1983)  shortest-path  sequence. 

One  of  the  most  important  theoretical  elements  of  ET  is  a special  kind  of  simple-to-complex  sequence.  The  whole 
notion  of  elaborative  sequence  is  based  on  a single  type  of  content  which  is  the  most  important  type  for  achieving  the  general 
goals  of  the  course.  ET  categorizes  the  content  of  the  course  with  three  types — concepts,  procedures,  or  principles.  When  an 
elaborative  sequence  has  been  done  to  each  of  them,  it  is  called  a conceptual  organization,  a procedural  organization,  and  a 
theoretical  organization  (Reigeluth  & Stein,  1983). 

SCM  provides  practical  guidelines  for  making  the  elaborative  sequence  for  procedural  and  theoretical  (transfer) 
organization.  The  SCM  process  begins  with  finding  conditions  which  make  a task  complex,  non-representative,  and  difficult 
for  learners.  Then,  it  constructs  an  epitome  which  is  simplified — an  easy  but  still  representative  version  of  the  task — by 
removing  or  dissolving  the  conditions  of  the  complex  task.  The  learners  will  start  learning  from  the  epitome  so  that  they  can 
work  with  the  epitome  without  experiencing  too  much  difficulty,  but  still  be  able  to  taste  the  representative  flavor  of  the 
whole  task.  Subsequently,  the  conditions  are  gradually  removed  or  dissolved  according  to  a preassigned  priority,  and  the  more 
complex  and  difficult  version  of  the  task  can  be  introduced.  In  this  manner,  the  whole  version  of  the  task  will  be  presented 
smoothly  and  meaningfully  (Kim  & Reigeluth,  1995a,  1995b). 

SCM  is  composed  of  two  major  parts — an  underlying  theory,  which  functions  as  a framework,  and  a process,  which 
is  used  as  a guideline  to  embody  the  underlying  theory. 

SCM  principles 

The  underlying  principles  of  SCM  fall  into  two  categories:  those  that  govern  epitomizing  and  those  that  govern  elaborating 
the  version  of  the  task.  If  epitomizing  can  be  compared  to  sketching,  then  elaborating  can  be  compared  to  adding  detail  to  the 
sketch.  So,  with  a good  epitome  and  good  elaborating,  it  is  possible  to  make  a sound  sequence  (Kim,  1994).  Figure  1 
illustrates  these  relationships. 

Epitomizing  principles.  The  principles  of  epitomizing  are  based  upon  the  notion  of  holistic  learning.  Therefore,  an  epitome 
should  begin  with 

• a whole  task  rather  than  a part  of  a task, 

• a simple  version  of  the  task, 

• a real-world  version  of  the  task  if  possible,  and 

• a representative  version  of  the  task. 

Elaborating  principles.  After  the  epitome  is  identified,  elaborating  entails  designing  each  subsequent  module  to  teach  another 
version  of  the  whole  task.  Each  elaboration  should  be  slightly 
• more  complex, 
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• more  divergent, 

• more  authentic,  and 

• less  typical. 

The  SCM  process 

While  the  principles  of  SCM  provide  conceptual  understanding  about  SCM  for  designers,  the  SCM  process  provides 
specific  guidelines  so  the  designers  can  apply  their  understanding. 

We  have  worked  with  several  versions  of  the  SCM  process  since  1991.  Since  the  version  outlined  in  List  1 was  the 
most  recent  version  at  the  time,  it  was  used  for  the  study. 

RESEARCH  PROBLEMS 

The  purpose  of  this  study  was  to  improve  the  principles  and  process  of  the  Simplifying  Conditions  Method  for  the  procedural 
tasks  by  using  formative  research  methodology  with  field  testing.  More  specifically,  this  study  investigated  the  following 
questions: 

• What  are  the  weaknesses  of  the  principles  of  SCM? 

• How  can  they  be  improved? 

• What  are  the  weaknesses  of  the  SCM  design  process? 

• How  can  they  be  improved? 

FORMATIVE  RESEARCH  FOR  INSTRUCTIONAL  THEORY 

The  major  research  methodology  for  this  study  is  the  formative  research  methodology  (Reigeluth,  1989)  which  is 
designed  to  improve  the  instructional  theory  and  model.  Using  the  formative  research  method  to  improve  instructional  design 
theory  was  a relatively  new  idea  until  Reigeluth  (1989)  suggested  it  as  a means  of  improving  prescriptive  instructional 
theory.  Since  then,  several  studies  have  used  this  methodology  for  that  purpose  (Clonts,  1993;  English,  1992;  Farmer,  1989; 
Kim,  1994;  Lim,  1994;  Roma,  1990;  and  Simmons,  1991).  These  studies  suggest  that  formative  research  methodology  is  an 
effective  tool  7or  improving  instructional  theories. 

Reigeluth's  (1989)  formative  research  methodology  is  based  on  the  principle  that  when  instruction  is  developed  under 
the  strict  guidance  of  a theory,  without  using  any  other  guidelines  or  input — not  even  the  designer’s  own  intuition — the 
instruction  (product)  is  an  ’’instance”  of  that  theory,  and  the  results  of  the  evaluation  of  the  instruction  will  reflect  the  theory’s 
weaknesses  and  strengths  and  will  point  to  ways  of  improving  the  theory. 

The  process  of  formative  research  for  instructional  theories  has  similarities  to  and  differences  with  formative 
evaluation  for  instructional  products.  Generally,  formative  evaluation  begins  when  the  designers  and/or  developers  of  the 
product  have  a basic  but  minimal  level  of  confidence  in  the  product  and  wish  to  determine  early  on  any  weaknesses  of  the 
product.  This  formative  evaluation  process  can  be  repeated  until  the  designer  is  satisfied  with  the  revised  product. 

The  process  and  purpose  of  formative  research  on  an  instructional  theory  or  model  are  similar.  An  instructional  theory  or 
model  requires  a significant  amount  of  trial  and  revision.  Formative  research  should  begin  only  when  the  creators  of  the 
instructional  theory  or  model  have  a basic  level  of  confidence  and  are  ready  to  examine  their  product  (the  theory  or  model)  for 
flaws.  Formative  research  is  complete  only  when  the  creators  are  satisfied  with  the  modified  version  of  the  theory,  which  is 
based  on  the  formative  research  results. 

Also,  there  is  a significant  difference  between  formative  evaluation  of  a product  and  formative  research  on  an 
instructional  theory.  In  the  formative  evaluation  of  an  instructional  product,  a designer  can  collect  data  on  the  product  from 
the  learners  directly  (see  Figure  2).  The  data>gathering  process  for  formative  research  on  an  instructional  theory  is  not  as 
simple,  however,  since  learners'  data  are  gathered  through  the  ’’instance”  of  the  theory  (see  Figure  3).  Consequently,  the 
validity  of  the  data  from  the  formative  research  for  the  theory  is  much  more  important  and  critical  than  those  of  the  formative 
evaluation  for  product.  For  the  formative  research,  the  ’’instance”  must  truly  represent  the  theory.  This  distinction  introduces 
following  salient  issues  which  should  not  be  overlooked  when  considering  the  validity  of  a formative  research  study  (Kim, 
1994). 

First,  since  the  instructional  designer  develops  the  "instance"  according  to  his  or  her  understanding  of  the  theory  or 
model,  the  designer’s  ability  to  apply  the  theory  is  one  of  the  critical  factors  in  research  validity.  If  the  designer  is  an  expert 
on  the  theory  or  model  and  has  considerable  experience  developing  "instances”  of  the  theory,  the  validity  of  the  study  will 
generally  be  acceptable.  However,  if  the  designer  lacks  significant  knowledge  and/or  experience  regarding  the  theory,  the 
validity  of  the  study  may  be  negated,  since  the  ’’instance”  may  not  accurately  reflect  the  theory  or  model. 


Second,  even  though  an  "instance"  can  be  determined  by  a theory  expert,  it  is  recommended  that  another  expert  on 
the  theory  confirm  this  "instance"  to  lend  further  credence  to  its  validity.  Consequently,  the  first  and  second  issues  can  be 
summarized  into  one  question — "Is  the  product  a true  instance  of  the  theory?"  This  raises  the  issue  of  construct  validity. 
Third,  when  the  product  is  "delivered"  to  the  learners,  the  learning  environment  should  be  as  natural  as  possible  in  order  to 
increase  the  external  validity  (Reigeluth,  1989).  It  is  also  necessary  to  determine  whether  the  characteristics  of  the  task  and 
the  learners  are  realistic  and  representative  the  instructional  settings  to  which  one  wishes  to  generalize. 

Fourth,  data  gathering  can  be  a critical  issue  in  formative  research  on  an  instructional  design  theory  or  model.  The 
following  questions  should  be  considered  in  the  planning  process:  "What  kind  of  data  will  be  really  useful  for  improving  the 
theory?"  and/or  "How  are  useful  data  gathered?"  (Reigeluth,  1983).  The  characteristics  of  the  instructional  design  theory  or 
model — sequencing,  selection,  instructional  strategies,  and/or  task  analysis — should  be  considered  when  answering  these 
questions. 

In  previous  formative  research  on  instructional  theories  or  models  (Clonts,  1993;  English,  1992;  Fanner,  1989; 

Kim,  1994;  Lim,  1994;  Roma,  1990;  and  Simmons,  1991),  data  gathering  relied  heavily,  but  not  exclusively,  on  the  use  of 
qualitative  data. 

DESIGN 

This  study  was  conducted  in  two  distinct  phases:  design  and  instruction.  In  the  first  phase,  we  used  the  SCM  process 
to  design  a course.  This  was  done  to  determine  the  weaknesses  of  the  SCM  process.  In  the  second  phase,  the  course  was 
taught  using  the  SCM  sequence.  This  was  done  to  determine  the  weaknesses  of  SCM  principles. 

FIRST  PHASE 

Task 

The  task  for  this  study  was  to  use  Authorware  Professional  to  create  a CAI  program.  It  was  selected  based  upon  the 
following  criteria: 

1.  The  task  should  be  procedural  or  theoretical,  so  that  the  procedural  or  theoretical  SCM  analysis  and  sequencing 
methodology  can  be  used. 

2.  There  should  be  a strong  interrelationship  among  the  topics  in  the  task.  (If  the  learning  task  were  composed  of  unrelated 
topics,  it  would  not  be  useful  for  this  study.) 

3.  The  context,  task,  and  audience  should  resemble  a normal  instructional  setting  as  much  as  possible  in  order  to  increase  the 
external  validity  (Reigeluth,  1989).  It  is  also  necessary  to  determine  whether  the  characteristics  of  the  task  and  the  learners  are 
realistic  and  representative  the  instructional  settings  to  which  one  wishes  to  generalize. 

4.  The  task  should  require  more  than  10  hours  to  learn,  including  the  time  required  for  in-class  activity  as  well  as  the  time 
required  to  complete  practice  exercises  and  homework.  If  the  task  is  short  and  requires  less  than  10  hours  to  master,  learners 
can  compensate  for  any  weakness  of  the  sequence  and  the  sequence  does  not  make  any  big  difference  on  the  effectiveness  and 
efficiency  of  learning. 

5.  There  should  be  a meaningful  interval  of  time  between  one  lesson  and  the  next  so  the  students  can  practice  and  review  what 
they  have  learned. 

Participants 

An  SCM  design  committee  of  six  members  was  established  to  design  the  SCM  sequence.  Their  specific  roles  were 
as  follows: 

1.  The  designer  designed  the  sequence  with  the  subject  matter  experts  (SMEs)  while  using  SCM. 

2.  The  two  SMEs  designed  the  sequence  with  the  designer  and  checked  the  content  validity  of  the  sequence  of  instruction. 

3.  The  peer-debriefer  gathered  data  while  observing  the  design  activity  of  the  designer  and  the  SMEs. 

4.  The  task  expert  checked  the  content  validity  of  the  designed  sequence  of  instruction  to  ensure  it  taught  the  task. 

5.  The  SCM  expert  checked  the  construct  validity  of  the  designed  sequence  of  instruction  as  an  SCM  sequence. 


Data  Gathering  and  Data  Analysis 

Data  were  collected  by  employing  three  different  processes:  (1)  self-monitoring  (Krieger,  1991 ) and  self-reflection 
(Schon,  1987)  by  the  designer;  (2)  the  observations  of  the  peer-debriefer  (Lincoln  & Guba,  1985);  and  (3)  debriefing  with  the 
designer,  peer-debriefer,  and  content  SMEs  who  participated  in  the  design  process. 

The  gathered  data  were  analyzed  mainly  by  triangulation  (Denzin,  1978;  Miles  & Huberman,  1984)  among  the  gathered  data 
from  the  three  processes:  designer's  self-monitoring  and  self-reflection  note,  peer  debriefer's  observation  note,  and  the 
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comments  from  the  debriefing  meetings.  More  specifically,  the  following  questions  were  answered:  What  results  are  similar 
among  the  data?  What  is  the  difference  among  data?  What  causes  the  differences  or  similarities? 

Procedure 

To  design  the  sequence  of  the  task  within  the  parameters  of  SCM,  the  designer,  one  SME,  and  the  peer-debriefer  met 
in  a quiet  place  at  the  SME’s  company.  While  designing  the  sequence  with  the  SME  according  to  the  SCM  process,  the 
designer  tried  to  note  in  detail  every  problem  or  question  that  arose.  The  peer  debriefer  also  tried  to  maintain  a complete 
working  log  for  each  step  of  SCM.  It  took  about  13  hours  total  over  three  days  to  complete  the  design  sequence.  After  each 
day's  work,  the  designer  cross-checked  the  gathered  data  with  the  peer-debriefer  and/or  the  SME. 

After  completing  the  sequence  design,  the  content  validity  and  the  construct  validity  of  the  sequence  were  checked  with  the 
task  (Authorware  Professional)  expert  and  SCM  expert. 

Results 

The  findings  from  the  first  phase,  according  to  the  identified  weaknesses,  could  generally  be  placed  into  four 
categories:  (1)  need  more  prescriptions,  (2)  need  to  consider  the  holistic  approach  more,  (3)  need  to  teach  SME,  and  (4)  need  to 
consider  job  aid/performance  of  designer/SME  (see  Table  1).  Each  weakness  was  also  analyzed  to  determine  whether  it  could 
be  strengthened  by  modifying  the  step(s)  or  criteria  or  by  adding  new  step(s)  or  criteria.  In  Table  1,  for  instance,  A.  1 1 and 
"M:  13"  from  the  first  column  indicate  that  1 1 more  steps  need  to  be  added  and  13  existing  steps  need  to  be  modified  with 
more  detailed  prescriptions. 

Discussion 

In  general,  the  SCM  process  works  fairly  well  and  successfully  reflects  the  principles  of  SCM.  However,  the  results 
of  this  study  clearly  show  why  the 

SCM  process  requires  further  improvement  and  what  needs  to  be  improved.  First,  issues  related  to  improving  the 
SCM  process  need  to  be  considered— is  it  really  possible  to  perform  task  analysis  and  sequencing  with  the  SCM  process? 
Second,  issues  related  to  specific  aspects  of  the  performance  of  SCM  designers  and  SMEs  also  need  to  be  considered — if  a 
designer  performs  SCM  for  the  first  time,  how  well  can  he  or  she  do  task  analysis  and  sequencing  with  SCM? 

Since  the  second  group  of  issues  is  more  related  to  training  or  training  materials  development  in  ISD  processes  and 
more  dependent  on  the  performance  of  designer,  the  original  focus  of  this  study  was  on  the  first  category — checking  the 
soundness  of  the  SCM  process  as  a design  process.  For  instance,  some  of  the  steps  from  the  "need  more  prescriptions" 
category  and  all  of  the  steps  from  the  "need  to  consider  holistic  approach"  category  focus  on  the  soundness  of  the  process.  If 
those  steps  were  not  added  or  modified,  the  vitality  of  the  SCM  process  would  be  questioned  as  an  instructional  design  theory. 
However,  most  of  the  steps  of  the  other  three  categories— the  rest  of  the  steps  from  "need  more  prescriptions"  and  all  of  the 
steps  from  "need  to  teach  SME"  and  "need  to  consider  job  aid/performance  of  designer/SCM"  focus  mainly  on  the  performance 
of  designers  and  SMEs.  Therefore,  those  steps  do  not  threaten  the  vitality  of  the  SCM  process.  Without  them,  however,  the 
efficiency  and  effectiveness  of  the  SCM  process  would  be  endangered.  Consequently,  in  order  to  improve  any  instructional 
design  theory,  both  aspects  must  be  considered  and  addressed. 

Issues  Concerning  the  Soundness  of  the  SCM  Process 

The  soundness  of  the  SCM  process  depends  on  the  consideration  of  two  issues:  (1 ) the  holistic  relationship  between 
the  principles  and  process  and  (2)  formative  evaluation  of  each  level  of  sequencing. 

Holistic  relationship  between  the  principles  and  process.  The  current  SCM  process  may  be  misunderstood  if  treated  as  a step- 
by-step  recipe-style  procedure.  Such  problems  can  occur  when  the  SME  asks  too  many  questions  or  demands  too  much 
deviation  from  the  SCM  process.  Excessive  deviation  is  risky  because  the  SCM  process  requires  a very  dynamic/recursive 
type  of  iteration,  a fine-tuning  process,  on  the  part  of  the  design  committee. 

In  fact  many  people  have  warned  about  the  potential  problem  of  the  overproceduralization  of  complex  ISD  processes 
(Davies,  1983;  Earle,  1985;  McCombs,  1986;  Shettle,  1983).  The  Elaboration  Theory  has  also  been  criticized  for  the 
possibility  of  overproceduralization  (Reigeluth,  1992;  Wilson  & Cole,  1992). 

In  order  to  forestall  such  problems,  the  SCM  process  must  be  expressed  in  a way  that  leaves  its  step-by-step 
prescriptions  more  holistically  integrated  with  its  principles.  This  can  be  done  in  two  ways.  First,  one  would  display  the 
relationship  between  the  principles  and  the  process  more  clearly  within  SCM  process  by,  for  instance,  integrating  a di-play  of 
the  related  principles  into  the  flowchart  representation  of  the  process.  Second,  one  would  emphasize  more  flexibility  in  the 
process  within  the  scope  of  the  principles,  so  that  the  design  activity  can  be  more  systemic  rather  than  systematic. 

Formative  evaluation  of  each  level  of  sequencing.  Formative  evaluation  is  recommended  for  each  level  of  sequencing.  The 
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scope  of  formative  evaluation  can  be  very  broad,  encompassing  one-on-one  evaluation,  small  group  evaluation,  and  field 
testing.  Consequently,  new  prescriptions  for  the  allowable  scope,  participants,  and  purpose  of  *he  formative  evaluation 
should  be  developed,  taking  into  consideration  the  rapid  prototyping  approach  such  as  the  following: 

1 . After  making  each  unit  of  the  sequence,  there  should  be  at  least  one  one-on-one  evaluation  of  the  unit. 

2.  For  the  one-on-one  evaluation  activity,  the  learner  should  be  realistic  and  representative  of  the  targeted  learners. 

3.  Evaluation  criteria  should  include  effectiveness,  efficiency,  and  appeal  of  the  instructional  unit. 

Issues  concerning  the  Performance  of  Designers  and  SMEs 

Although  this  study  made  separate  efforts  to  find  weaknesses  in  the  principles  and  process  of  SCM,  these  two 
elements  are  holistically  interrelated.  Within  this  interrelationship,  the  success  of  SCM  principles  is  relatively  dependent  on 
how  well  the  designers  understand  SCM,  while  the  success  of  the  SCM  process  is  relatively  dependent  on  the  performance  of 
the  designers  and  SMEs,  which  is  based  on  this  understanding  and  training.  Consequently,  the  success  of  the  principles 
(understanding)  and  the  success  of  the  process  (performance)  are,  taken  together,  necessary  and  sufficient  conditions  for  the 
success  of  SCM. 

Viewed  from  this  perspective,  it  appears  that  the  current  version  of  SCM  might  be  improved  by  considering  the 
performance  and  understanding  of  designers  and  SMEs  on  the  following  points: 

• The  SCM  process  needs  to  be  described  in  plainer  and  more  explicit  terms. 

• The  available  criteria  are  insufficient  to  evaluate  the  success  of  some  steps. 

• Easy  concrete  examples  of  the  SCM  approach  would  enhance  the  understanding  and  performance  of  the  SMEs  and 

the  designers. 


SECOND  PHASE 

In  the  second  phase,  using  the  product  of  the  first  phase's  design  activity,  the  designer  taught  the  workshop  for 
instructional  designers  at  the  Computer  Education  Research  Center  (CERC)  of  the  Korean  Education  Development  Institute 
(KEDI)  in  Seoul,  Korea.  The  designer  also  gathered  data  from  interactive  interviews  with  participants  of  the  workshop  and 
observed  the  class.  Likert-style  data  on  their  attitudes  were  gathered  from  the  learners.  All  gathered  data  were  checked  and 
analyzed  by  triangulation  with  other  learners. 


Method 

Participants 

There  were  10  participants  for  the  second  phase.  Since  the  second  phase  was  an  unofficial  internal  workshop  of  a 
research/development  institute  which  specializes  in  research  and  production  of  computer-based  instruction,  all  of  the 
participants  were  instructional  designers  employed  by  the  institute.  Each  person  had  designed  about  five  CAI  programs  every 
year  and  had  worked  at  the  institute  for  at  least  two  years  as  an  instructional  designer.  Some  participants  were  fluent  in  using 
some  authoring  systems/languages,  but  none  had  had  any  prior  experience  with  Authorware  Professional. 


Data  Gathering 

While  conducting  the  workshop,  the  instructor  gathered  data  through  personal  interviews  with  participants, 
assignments,  observation,  and  the  instructor's  personal  reflections.  After  completing  the  workshop,  the  instructor  held  three 
debriefing  meetings  with  small  groups  of  participants. 

Procedure 

The  second  phase  included  three  stages — initiating,  implementing,  and  debriefing. 

Initiating  stage.  Before  starting  the  workshop,  the  instructor  interviewed  every  participant  to  assess  the  participa"t's 
previous  experiences.  During  the  interview,  the  instructor  explained  the  purpose  of  this  study  and  encouraged  each  participant 
to  provide  honest  feedback  on  the  sequence  of  the  workshop. 

Implementing  stage.  The  instructor  taught  the  nine-day  workshop  without  any  other  instructors  or  assistants.  Each 
day  of  the  workshop  included  a lecture  lasting  about  one  hour  and  then  individual  practice  for  at  least  one  hour.  Two 
assignments  were  given  during  the  nine  days’  training,  and  each  took  about  one  and  a half  hours  to  complete.  During  the 
workshop,  the  instructor  interviewed  each  participant  at  least  twice  to  obtain  meaningful  and  relevant  information  about  the 
sequence  of  the  workshop.  During  the  interim  interviews,  the  learners  were  asked  the  following  questions: 
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• What  did  you  like  or  dislike  about  today’s  activity?  Is  your  answer  related  to  the  sequence  of  instruction  (or 

ordering  and  grouping  of  instruction)? 

• What  do  you  think  would  enhance  your  motivation  in  the  course?  Is  it  related  to  the  sequence  of  instruction? 

• What  would  help  you  understand  the  task  better?  Is  your  answer  related  to  the  sequence  of  instruction? 

• If  you  encountered  difficulties  related  to  the  sequence  of  instruction  while  learning  the  lesson,  how  could  those 

difficulties  be  avoided  in  the  future? 

• Is  the  topic  too  difficult?  Is  this  because  of  the  sequencing? 

• As  a learner  in  this  course,  how  do  you  think  the  sequence  can  be  changed  to  make  the  course  more  attractive? 

After  the  last  day  of  the  workshop,  the  learners  were  asked  to  respond  to  a final  attitude  survey  to  assess  the  appeal  of 

the  instruction  and  the  their  attitudes  toward  the  sequence  of  instruction.  The  survey  was  composed  of  13  Likert-style 
questions  and  five  open-ended  questions,  which  asked  the  learners  if  they  liked  the  sequence  of  the  daily  lesson.  Table  2 
summarizes  the  survey  results. 

Debriefing  stage.  After  the  workshop,  a debriefing  meeting  was  attended  by  a debriefer  for  the  first  phase,  two 
learners,  and  the  instructor.  In  the  meeting,  the  instructor  shared  the  findings  from  his  observations  and  personal  interviews 
and  then  asked  some  questions  which  had  been  suggested  by  these  personal  interviews  and  observations. 

Data  Analysis. 

Once  data  collection  was  complete,  the  information  was  organized  ' o that  comparing  the  difference  and  similarity 
among  the  data  sets  could  be  performed.  The  most  important  data  were  from  the  interviews  with  the  learners  and  from  the 
observation  record.  Like  the  data  from  the  first  phase,  the  interview  results  from  the  second  phase  were  analyzed  with  an 
emphasis  data  on  among  the  learners’  and  instructor's  observations.  Our  data  analysis  involved  data  reduction,  data  display, 
and  drawing  conclusion  (Miles  & Huberman,  1984). 

This  analysis  helped  us  identify  the  major  strengths  and  weaknesses  of  the  sequence  so  that  the  theory  could  be 
improved.  The  findings  of  the  first  and  second  phases  were  used  to  support  and  develop  conclusions  that  might  help  determine 
the  underlying  principles  of  SCM  and  the  weaknesses  of  the  SCM  process. 

Results 

Generally,  the  data  clearly  show  the  strength  of  the  SCM  sequence.  The  general  results  of  the  open-ended  questions 
also  agreed  with  those  of  the  Likert-style  questions,  in  supporting  the  strength  of  the  SCM  approach. 

Table  2 displays  a summary  of  the  attitude  survey,  which  is  composed  of  three  major  groups:  (1)  general  attitude 
toward  the  workshop,  (2)  feedback  on  the  epitome,  and  (3)  feedback  on  the  sequence  of  the  workshop. 

General  attitude  toward  the  workshop:  The  results  from  questions  1 to  5 display  the  learners'  general  attitude  toward 
the  workshop.  For  the  most  part,  the  learners  were  very  satisfied  with  the  workshop.  The  mean  of  the  answers  to  questions  1 
to  4 was  between  1 and  2,  which  stands  for  "strongly  agree"  and  "agree."  Also,  the  mean  for  question  5 was  4.5,  indicating 
that  all  of  the  learners  felt  they  had  successfully  understood  the  main  characteristics  of  the  task. 

Feedback  on  the  epitome:  The  feedback  on  question  6 shows  that  the  first  lesson  (epitome)  was  highly  helpful  to  the 
learners  in  understanding  the  course  as  a whole.  The  mean  of  question  6 was  1 .9,  indicating  that  the  learners  agreed  that  the 
first  lesson  was  very  helpful  in  understanding  the  content  of  the  course  as  a whole. 

Question  7 originated  from  a learner's  suggestion  in  an  interim  interview  during  the  workshop.  That  person  had 
some  understanding  of  the  Elaboration  Theory  and  liked  the  idea  of  "epitome."  He  felt  that  if  the  epitome  could  be  reviewed 
during  the  course,  it  would  help  the  learners  understand  the  relationship  between  the  epitome  and  each  elaborated  lesson,  thus 
encouraging  better-structured  understanding  while  allowing  them  to  verify  their  progress.  The  learner  suggested  this  idea  in  the 
middle  of  the  workshop.  Even  so,  it  is  interesting  that  this  person  did  not  strongly  disagree  to  question  7 when  it  was  asked 
after  the  workshop.  On  his  survey,  he  noted  that  "even  if  it  is  a good  idea,  it  may  not  be  practical,  since  it  will  take  a lot  of 
time." 

Feedback  on  the  sequence  of  the  workshop.  Results  for  questions  8 and  10  show  consistently  that  the  sequence  of  the 
course  was  not  very  difficult  and  that  learning  one  lesson  facilitated  the  mastery  of  the  next.  The  results  from  questions  1 1 , 

12,  and  13  clearly  show  that  the  sequence  of  the  course  was  very  good.  The  generally  positive  responses  to  Question  10 
support  this  conclusion. 

The  results  of  the  personal  interviews  support  the  findings  from  the  attitude  survey.  The  questions  and  the  summary 
of  answers  are  as  follows: 

• What  did  you  like  or  dislike  about  the  course?  Is  your  answer  related  to  the  sequence  of  instruction  (or  ordering  and 
grouping  of  instruction?):  Almost  all  of  the  participants  liked  the  first  lesson  and  sequence.  One  person  noted  that  repetition 
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of  the  content  of  the  first  lesson  (epitome)  during  the  first  three  lessons  was  a little  boring;  however,  it  proved  very  helpful 
for  later  learning  for  him. 

• What  do  you  think  enhanced  your  motivation  in  the  course?  Is  your  answer  related  to  the  sequence  of  instruction?: 
"Since  the  first  lesson  was  practical,  it  was  not  very  difficult  to  maintain  motivation.  Each  time  we  learned  a limited  amount 
of  content,  and  there  was  always  the  potential  for  elaboration.  Because  of  this  potential,  it  was  possible  to  peek  at  the  content 
of  upcoming  lessons  and  this  enhanced  my  motivation." 

• What  would  help  you  to  understand  the  task  better?  Is  the  answer  related  to  the  sequence  of  instruction?:  "It  might 
be  helpful  to  check  where  the  current  lesson  is  on  the  big  map  of  the  whole  task." 

• If  you  encountered  difficulties  related  to  the  sequence  of  instruction  while  learning  the  lesson,  how  could  those 
difficulties  be  avoided  in  the  future?:  "Using  window  was  one  of  the  prerequisite  skills  for  this  workshop.  Since  I didn't  know 
anything  about  the  window,  I had  a lot  of  stress  with  window  even  though  the  class  activity  did  not  require  using  window  a 
lot.  It  might  be  beneficial  to  have  enough  time  to  practice  the  prerequisite  skills  until  I feel  confident  before  starting  any 
major  lesson." 

• As  a learner  for  this  course,  how  do  you  think  the  sequence  can  be  changed  to  make  the  course  more  attractive?:  "In 
general,  the  sequence  of  this  course  was  OK  for  me.  However,  since  my  learning  style  is  more  inquiry-based,  I would  like  to 
have  a chance  to  explore  more,  rather  than  following  the  given  instruction." 

Discussion 

In  light  of  this  study's  goal — finding  weaknesses  so  we  could  improve  the  underlying  theory  of  SCM — the  second 
phase  was  unproductive,  since  no  significant  weakness  was  found.  This  was  expected,  however,  since  the  results  of  the  pilot 
studies  were  very  similar  to  those  of  the  second  phase.  Consequently,  rather  than  identifying  weaknesses,  the  second  phase 
provided  some  evidence  of  the  strength  of  the  underlying  theory  of  SCM.  Basically,  it  confirmed  the  validity  of  SCM's 
approach  of  providing  a big  picture  of  the  task  early  in  the  instruction,  thus  enriching  the  epitome  with  additional  elaboration 
efficiently  and  effectively,  while  maintaining  the  learners'  motivation  at  a high  level  throughout  the  course. 

Besides  indicating  these  strengths,  die  results  of  the  second  phase  also  support  the  following  two  recommendations 
regarding  possible  improvements  to  the  underlying  principles  of  SCM  and  another  one  regarding  the  SCM  process  and  SCM 
principles. 

1.  This  study  revealed  a need  to  improve  the  descriptions  of  the  principles  identifying  the  SCM  and  the 
interrelationships  among  those  principles  by  ensuring  such  descriptions  are  concrete,  clear,  and  self-explanatory.  As  things 
now  stand,  designers  must  take  a great  deal  of  time  (inventing  many  explanations  and  analogies  on  their  own)  to  introduce  the 
principles  to  the  SME(s).  If  the  principles  are  to  be  understood  easily,  workable  explanations  will  have  to  be  in  place 
beforehand.  One  example  would  be  calling  the  first  day’s  instruction  the  "epitome." 

2.  The  current  way  of  describing  the  principles  provides  a kind  of  absolute  set  of  criteria,  such  as  "a  whole  task  rather 
than  a part  of  a task"  (the  first  epitomizing  principle).  But  in  practice,  the  definition  of  a whole  task  can  be  varied  according 
to  the  characteristics  of  the  learners,  the  delivery  constraints,  etc.  For  instance,  when  the  designer  explained  about  the  first 
epitomizing  principle  to  the  SME,  the  SME  was  asked  about  the  scope  of  the  course  and  other  practical  limits  of  the  course. 
This  dialog  helped  to  identify  the  epitome  version  of  the  task. 

Thus,  it  would  be  better  to  provide  designers  and  SMEs  with  formulations  of  the  principles  that  take  these  variable  conditions 
into  account.  For  instance,  the  first  epitomizing  principle  (a  whole  task  rather  than  a part  of  a task)  could  be  described  as  "a 
whole  task  rather  than  a part  of  a task,  within  the  limits  imposed  by  the  characteristics  of  the  learners,  their  needs,  the  goal  of 
the  course,  and  the  delivery  constraints."  In  this  manner,  the  relationship  between  the  first  epitomizing  principle  and  the 
preparation  stage  of  the  SCM  process  can  also  be  clarified. 

3.  The  constructivist  approach  should  be  considered  for  the  micro-level  design.  Most  of  the  learners  liked  the  SCM 
sequence;  however,  about  20%  of  learners  wanted  to  have  different  ways  of  studying,  such  as  explorative  or  constructive 
studying,  made  available  to  them.  These  results  can  easily  mislead  one  into  concluding  that  SCM  limits  the  learners'  learning 
style  to  expository  only.  However,  SCM  does  not  limit  its  application  with  any  one  way  of  learning  such  as  expository  or 
discovery.  SCM  can  be  used  for  discovery  or  in  an  explorative  way  also  because  SCM  determines  the  learning  sequence  at  the 
macro  level  and  the  decision  of  whatever  to  use  an  expository  or  explorative  way  of  learning  is  made  at  the  micro  level.  So 
these  comments  do  not  reflect  on  the  SCM  but  rather  on  the  decision  of  the  designer  at  the  micro  level.  For  these  studies  of 
the  SCM  process,  we  implicitly  used  only  the  expository  way  of  teaching. 

These  results  raise  an  issue — prescriptions  need  to  be  added  to  micro-level  design  which  can  be  designed  for  either 
expositive  or  explorative  learning. 
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CONCLUSION 


The  results  of  this  study  suggest  two  conclusions.  First,  the  current  SCM  process  is  workable;  however,  three 
recommendations  were  found  for  improvement. 

• The  need  to  stress  the  holistic  rather  than  the  step-by-step  approach 

• The  need  to  add  more  detailed  prescriptions. 

• The  need  for  formative  evaluation. 

Second,  we  were  unable  to  find  any  critical  weaknesses  that  might  lead  to  possible  improvements  in  the  SCM 
principles,  since  most  of  the  learners  were  very  satisfied  with  the  sequence  of  the  instruction.  The  results  of  this  study  reveal 
that  even  though  the  principles  of  SCM  are  fundamentally  sound,  there  is  still  room  for  improvement  as  indicated  above. 
Therefore,  future  research  can  focuse  more  on  the  application  of  SCM  and  can  confront  such  challenges  as  finding  the  best 
ways  to  train  and  guide  the  novice  designer. 
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Figure  I:  The  underlying  principles  of  SCM 


List  1:  Process  for  SCM  procedural  analysis  and  sequencing  (1993  version). 

The  SCM  Process  for  Procedural  Analysis  and  Sequencing 
(All  steps  are  performed  by  a designer  and  a subject  matter  expert.) 

Prepare 

1)  Establish  rapport  with  subject  matter  expert  (SME). 

2)  Identify  the  characteristics  of  the  task  in  general. 

3)  Identify  the  characteristics  of  the  learner  in  general. 

4)  Identify  the  delivery  constraints  of  the  task  in  general. 

5)  Introduce  the  SME  to  SCM. 

Design  the  Epitome 

1)  Identify  and  order  the  simplifying  conditions. 

1.1)  Identify  the  full  variety  of  versions  of  the  task. 

1 .2)  Describe  the  conditions  which  differentiate  each  version. 

1 .3)  Rank  order  the  conditions  according  to  representativeness  of  the  whole  task. 

2)  Identify  the  epitome  version  of  the  task. 

2.1)  Recall  the  simplest  version  that  is  representative. 

2.2)  Describe  the  conditions  for  the  epitome  version. 

3)  Analyze  the  epitome  version. 

3.1)  Decide  whether  the  task  is  procedural  or  theoretical.  If  it  is  procedural,  continue. 

3.2)  Identity  all  major  steps  for  performing  the  epitome  version 

under  the  conditions. 

3.3)  Analyze  each  step  down  to  the  entry  level  of  description  (hierarchical  analysis). 

3.4)  Draw  a flowchart  for  the  epitome  version. 

4)  Analyze  supporting  content  for  the  epitome. 

5)  Check  the  size  of  the  epitome. 

5.1)  Analyze  the  delivery  time  constraints  of  the  specific 

learning  situation,  if  any. 

5.2)  Compare  the  size  of  the  epitome  to  the  time  constraints. 

5.3)  If  the  epitome  is  longer  than  the  time  constraints,  reduce  its  size 

by  adding  another  simplifying  condition. 

5.4)  If  the  epitome  is  much  shorter  than  the  time  constraints,  increase  its  size  by  removing  one  of  the  simplifying 
conditions. 

6)  Design  and  develop  instruction  for  the  epitome. 
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Prepare  for  Elaboration 

1)  Refine  the  simplifying  conditions. 

1.1)  Identify  additional  simplifying  conditions,  if  any,  that  may  have  beer  overlooked. 

1.2)  Check  for  any  conflict  or  overlap  among  the  conditions  and  adjust  them. 

1.3)  Rank  order  the  simplifying  conditions. 

2)  Identify  the  relative  amount  of  learning  that  would  be  required  by  removing  each  simplifying  condition. 

3)  Identify  those  simplifying  conditions  whose  removal  will  require  learning  skills  similar  to  those  for  another  simplifying 
condition. 

Design  the  Elaborative  Sequence 

1)  Identify  the  next  simplest  version  (the  next  elaboration). 

1.1)  According  to  the  rank  order,  select  the  next  simplifying  condition  to  remove. 

1.2)  If  the  removal  requires  additional  conditions,  identify  them  and  add  them  to  the  rank  order. 

1.3)  Identify  the  version  of  the  task  that  meets  the  new  conditions. 

2)  Analyze  the  version. 

2. 1 ) Identify  all  major  steps  for  performing  the  simplest  version  under  the  conditions. 

2.2)  Analyze  each  step  down  to  the  entry  level  of  description,  considering  all  previous  instruction. 

2.3)  Draw  a flowchart  for  the  lesson. 

3)  Analyze  the  supporting  content  for  the  version. 

4)  Check  the  size  of  the  version. 

4.1)  Compare  the  size  of  the  lesson  to  the  time  constraints. 

4.2)  If  the  lesson  is  longer  than  the  time  constraints,  add  in  a secondary  simplifying  condition  and  return  to  step  4. 1 . 

4.3)  If  the  lesson  is  much  shorter  than  the  time  constraints,  remove  another  simplifying  condition  according  to  the  rank 
order  and  return  to  step  4. 1 . 

5)  Design  and  develop  instruction  for  the  lesson. 

6)  Repeat  steps  1 through  5 for  each  remaining  simplifying  condition. 

Figure  2:  The  Formative  Evaluation  Model  for  an  instructional  product. 
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Figure  3:  The  Formative  Research  Model  for  an  instructional  theory. 


Table  1 Summary  of  the  Results  of  the  First  Phase 


Weakness 

Need 

more 

prescrip- 

tions 

Need  to 
consider 
holistic 
approach 

Need  to 

teach 

SME 

Need  to  consider  job 
aid/performance  of 
designer/SME 

a 

A:  11 

A:  0 

A:  10 

A:  3 

C7* 

K 

Ph 

M:  13 

M:  5 

M:  1 

M:  1 

M:  Need  to  modify  the  step(s)  or  criteria 
A:  Need  to  add  new  stents'!  or  criteria 
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Table  2:  Workshop  Survey  Results 


No. 

Question 

SA 

A 

u 

D 

SD 

1 

I learned  a lot  from  this  course. 

9 

i 

0 

0 

0 

2 

This  course  was  very  interesting. 

8 

2 

0 

0 

0 

3 

I really  liked  taking  part  in  this  course. 

8 

2 

0 

0 

0 

I 

4 

I'm  willing  to  participate  again  if  a higher  level  of 
this  course  is  offered  someday. 

4 

6 

0 

0 

0 

5 

I still  don't  know  what  the  main  characteristics  of 
the  task  presented  in  this  course  are. 

0 

0 

0 

5 

5 

6 

The  first  lesson  was  very  helpful  in  understanding 
the  content  of  the  course  as  a whole. 

3 

5 

2 

0 

0 

n 

7 

It  would  be  very  good  if  the  first  lesson  were 
reviewed  during  this  course,  in  between  the 
presentations  of  new  material. 

0 

2 

4 

3 

1 

8 

This  course  was  too  difficult  for  me. 

0 

0 

0 

4 

6 

9 

The  amount  of  learning  for  each  day  was  too  much 
for  me. 

0 

1 

2 

6 

1 

10 

It  was  not  very  difficult  to  understand  the  content  of 
each  new  lesson,  since  each  lesson  was  closely 
related  to  the  previous  one. 

3 

6 

1 

0 

0 

m 

11 

The  sequencing  of  topics  from  day  to  day  was 
handled  very  well. 

6 

4 

0 

0 

0 

12 

The  content  was  not  presented  in  a logical  sequence. 

0 

0 

0 

1 

9 

13 

It  would  have  been  better  if  the  whole  sequence  of 
each  lesson  had  been  composed  differently. 

0 

0 

0 

4 

6 

SA:  Strongly  Agree,  A:  Agree,  U:  Undecided,  D:  Disagree,  SD:  Strongly  Disagree 

I:  General  attitude  on  the  workshop 

II:  Feedback  on  the  epitome 

HI:  Feedback  on  the  sequence  of  the  workshop 
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Recent  developments  in  hypermediated  software  suggest  new  challenges  for  instructional  designers  of  interactive 
multimedia.  Product  enhancements  provide  authoring  software  with  increased  capabilities  for  designing  and  producing 
multimediated  instruction.  The  enhancements  permit  new  possibilities  for  the  appearance  and  functionality  of 
courseware,  and  thus  pose  new  challenges  for  instructional  designers. 

Further,  the  increasing  ease  with  which  the  improved  authoring  software  can  potentially  be  used,  could  lead  to  a situation 
in  which  people  produce  instructional  courseware  without  the  aid  or  advice  of  instructional  designers.  Products  that 
result,  whether  stand-alone  desktop  systems  or  those  that  incorporate  other  resources  such  as  material  from  the  world  wide 
web,  have  vast  potential.  Yet  the  new  technological  enhancements  will  not  necessarily  be  applied  in  a way  that  truly 
improves  the  instruction. 

Technology  offers  “bells  and  whistles"  that  seem  to  have  great  potential  but  often  take  the  lead  and  interfere  with 
instructional  design.  Indeed,  if  misapplied,  the  software  enhancements  could  actually  befuddle  the  learning  process.  A 
framework  is  needed  to  provide  guidance  for  people  who  must  produce  instructionally  sound  products  that  utilize  a mix  of 
stagnant  text,  text  as  a dynamic  visual  image,  still  and  motion  visual  images,  and  audio  within  a hypermediated  format. 
Such  a framework  can  provide  the  necessary  guidance  to  make  wise  use  of  the  new  enhancements  to  the  technology. 

The  processes  of  linking  visual  images  with  visual  text,  and  making  connections  between  realia  and  the  meaning  of  the 
images  in  a hypermediated  format,  lead  to  questions  about  effective  instructional  design  of  courseware  that  incorporates 
those  processes.  Moreover,  those  processes  cause  software  evaluators  to  rethink  the  standard  for  effective  and  appropriate 
presentations. 

This  paper  addresses  some  of  the  challenges  and  dilemmas  faced  by  two  instructional  designers  as  they  incorporated  visual 
imagery  and  audio  components  into  the  design  of  two  distinct  interactive  multimedia  products.  Each  instructional 
product  was  designed  as  a professional  development  seminar  or  series  of  instructional  seminars  for  two  distinctly  different 
groups  of  adult  learners.  One  product  was  designed  for  preservice  teachers  who  spend  a significant  amount  of  time  on 
campus  and  in  classes,  while  the  other  was  designed  for  practicing  social  workers  who  are  employed  in  isolated,  rural 
areas  where  access  to  professional  development  is  extremely  limited. 

The  multimedia  products  present  content-specific  information  pertinent  to  learners  through  the  use  of  visual  text,  still 
images,  still  and  animated  graphics,  sound,  and  full-motion  video  vignettes  contained  on  a laser  videodisc  or  CD-ROM. 
The  two  programs  provide  learners  with  new  information,  observational  examples,  and  interactive  practice  about  content 
specific  to  their  fields,  either  teaching  or  social  work.  There  are  many  similarities  shared  between  the  instructional  needs 
of  the  two  products  and  so  many  design  aspects  are  common  to  both,  however  the  graphical-user  interfaces  between  the 
two  programs  are  very  different. 

The  teacher's  product  contained  content  about  the  National  Council  of  Teachers  of  Mathematics'  Standards  (NCTM, 

1991).  Students  normally  would  cover  this  material  in  class  as  part  of  their  regular  coursework  and  through  other 
techniques  of  instruction.  The  social  worker's  product  differs  in  purpose.  It  offers  ten  different  topics  of  study  that  are 
not  addressed  within  the  course  of  regular  university  preparation  for  social  workers.  The  topics  were  determined  by 
conducting  a needs  assessment  with  practicing  social  workers. 

The  social  worker's  program  is  quite  extensive  and  is  being  developed  as  a series  of  ten  separate  modules  with  similar 
functionality  to  accommodate  people  who  have  very  limited  computer  skills.  It  uses  standard  design  elements  such  as 
text-based  menus,  buttons,  pop-up  text  boxes,  concept  maps,  and  the  choice  of  receiving  audio-based,  text-based,  or  a 
combination  of  audio  and  text  within  the  instructional  interface. 

The  teacher's  program  replaces  text  menuing  with  a graphical  user  interface  (GUI)  that  presents  familiar  objects  within  a 
traditional  school  setting.  The  main  menu  is  a graphic  of  a school  hallway  lined  with  doors  and  an  exit  sign.  Each  door 
leads  to  a room  that  contains  desks,  books,  and  chalkboards  to  access  information.  This  program  also  contains  an 
electronic  notebook  in  which  the  learners  can  write  their  questions  and  reactions  to  their  observations.  Both  programs  are 
hypermediated,  employ  video  vignettes,  and  contain  a presentation  component,  an  application  component,  and  an 
assessment  component. 
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Both  product  design  teams  were  able  to  follow  some  common  theoretical  elements  of  interactive  system  design,  yet  both 
broke  new  ground  in  their  own  way  as  they  strove  to  create  a product  best  suited  to  their  given  audience  and  instructional 
needs.  Formative  evaluation  was  conducted  on  both  programs  (Clark,  1995;  Knupfer,  Barrett,  & Lee,  1995).  In  addition, 
summative  evaluation  was  conducted  on  the  product  designed  for  teachers  (Clark,  1995).  Results  of  the  summative 
evaluation  of  the  teacher’s  multimedia  product  indicate  that  significant  differences  occurred  in  the  abilities  of  teachers  who 
used  this  program  to  identify,  understand,  and  explain  specific  teaching  strategies  they  observed  in  the  video  vignettes  as 
compared  to  those  who  did  not  use  the  multimedia. 

The  product  designed  for  social  workers  is  much  more  extensive;  it  represents  a series  of  ten  distinct  topics  of  study  that 
each  carry  three  continuing  education  units  of  credit.  Each  topic  is  considered  as  a unique  course  of  study  and  is  contained 
in  a separate,  but  coordinated  module  like  one  title  in  a series  of  books  written  by  different  authors  but  with  a single 
series  editor.  Although  the  ten  modules  are  designed  as  a set  of  training  for  social  workers,  the  modules  can  be  studied  in 
any  order  and  in  any  amount.  The  criteria  for  receiving  credit  for  each  module  rests  in  completion  of  an  on-line  test  with 
a passing  grade. 

The  social  work  modules  are  being  phased  into  numerous  communities  as  the  development  progresses  and  eventually  all 
ten  modules  will  be  in  place  in  numerous  locations.  As  each  module  of  the  series  is  developed,  it  goes  through 
formative  evaluation  and  field  testing.  Summative  evaluation  will  be  conducted  after  several  modules  have  been  placed 
into  the  social  work  offices  in  various  communities.  This  paper  limits  discussion  to  the  first  module.  Child 
Development , for  the  sake  of  simplicity  and  clarity. 

Success  of  any  product’s  effectiveness  will  be  affected  by  the  learners’  acceptance  or  attitude  toward  the  product,  the 
product’s  match  with  learners’  needs  both  in  a technical  sense  and  in  terms  of  content,  and  the  product’s  ability  to  be  used 
within  a given  environment.  Therefore  any  evaluation  must  also  consider  the  reactions  of  both  the  social  workers  and 
preservice  teachers  to  using  the  hypermedia.  Results  of  the  evaluations  to  date  suggest  some  guidelines  that  instructional 
designers  should  consider  when  designing  such  products.  Further,  as  we  think  about  ways  to  improve  those  products  and 
enhance  their  content  and  functionality  with  material  from  outside  sources,  such  as  those  from  the  internet,  we  need  to  be 
able  to  evaluate  the  quality  of  the  outside  resources  as  well. 

Interactive  multimedia  computer  tools  can  expand  our  ways  of  thinking  and  perhaps  encourage  metacognition,  more 
deeply  than  software  that  contains  only  standard  text,  few  images,  or  non-interactive  video.  In  interactive  multimedia 
contexts,  learners  browse  and  search  through  video,  images  and  text;  form  interpretive  thoughts;  make  connections 
between  visual  text,  still  images,  and  full-motion  video  images;  and  cluster  discrete  fragments  of  information  into 
meaningful  ideas.  The  challenge  for  instructional  designers  is  to  take  the  set  of  guidelines  that  has  been  developed  for 
interactive,  computer-based  instruction,  and  extend  them  to  include  guidelines  that  consider  the  dynamic  nature  of  the 
myriad  possibilities  that  become  available  with  new  technological  tools,  resources,  and  techniques. 

As  authoring  tools  become  more  transparent  and  more  people  become  skilled  at  incorporating  the  internet  into  their 
work,  the  volume  of  interactive,  multimediated  software  intended  for  self  instruction  is  likely  to  increase.  This  increase 
signals  two  things:  more  software  that  is  produced  by  novice  instructional  designers  or  people  who  have  limited 
knowledge  of  techniques  appropriate  to  efficient  learning,  and  the  need  for  guidelines  to  use  in  the  production  and 
evaluation  of  the  hypermediated  software.  Such  a design  framework  will  go  beyond  the  guidelines  used  for  computer- 
based  instruction  as  we  knew  it  a few  years  ago  and  if  not  able  to  provide  succinct  answers,  will  need  to  at  least  address 
additional  areas  of  possibilities. 

Designing  hypermediated  instruction  is  a formidable  process.  It  calls  for  deep  analysis  of  many  interconnected  and 
dynamic  components,  and  results  in  a set  of  organized  activities  and  processes  (see  Andrews  & Goodson,  1980,  and 
Gustafson  & Powell,  1991).  One  critical  component  of  hypermediated  instruction  is  the  use  of  visual  imagery.  Knupfer 
has  suggested  guidelines  for  employing  visual  imagery  within  computer-based  local  and  distance  instruction  that 
incorporate  some  of  the  well-established  standards  for  instructional  television  production  (1994a;  1995).  There  is  a need 
to  take  that  set  of  guidelines  and  extend  it  even  further.  In  doing  so,  the  following  questions  seem  essential  to  any 
improvement  of  instructional  design  guidelines  for  interactive  multimedia.  Their  answers  help  to  construct  a possible 
framework  to  guide  future  designs  for  hypermediated  learning: 
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• How  should  the  visual  elements  of  hypermedia  be  structured? 

• What  are  the  proper  uses  of  visual  metaphors? 

• What  are  the  formidable  questions  inherent  in  the  presentation  of  content  and  images? 

• When  does  design  supersede  content? 

• What  is  the  importance  of  the  instructional  designer’s  familiarity  with  new  models  of  instruction  and  learning  theory? 

• What  theoretical  elements  are  appropriate  for  the  graphical-user  interface? 

• Are  the  images,  language,  and  content  free  of  cultural  and  gender  bias? 

• How  do  the  images,  language,  and  content  accurately  represent  the  cultural  and  gender  variables? 

• How  should  formative  and  summative  evaluation  occur? 

Although  there  are  no  answers  that  will  work  in  all  situations,  for  all  audiences,  there  are  some  considerations  that 
designers  can  incorporate  into  the  planning  process.  Further,  as  hypermedia  takes  fuller  advantage  of  on-line  resources  it 
will  be  important  for  designers  to  consider  strategies  for  handling  dynamic  information  and  images  from  outside  sources, 
such  as  the  internet. 

CONSIDERATIONS 

How  should  the  visual  elements  of  hypermedia  be  structured? 

Visual  imagery  is  one  of  the  most  important  elements  of  hypermediated  instruction,  yet  many  designers  fail  to 
incorporate  it  to  it  fullest  potential.  If  used  wisely  the  visuals  can  greatly  enhance  the  instructional  message.  Like  other 
elements  within  the  product  design,  visual  information  needs  to  be  applied  in  a consistent  manner  yet  with  attention  to  a 
comfortable  blend  of  variety  and  aesthetics.  Both  of  the  products  evaluated  for  this  study  did  an  excellent  job  of  applying 
visual  elements  to  a certain  extent,  and  both  have  areas  in  which  they  can  be  improved. 


The  greatest  weakness  of  both  products  stemmed  from  the  limited  experience  of  certain  key  decision  makers  who  lacked 
enough  skill  to  visualize  elements  of  the  learning  process.  Having  not  been  trained  in  instructional  design,  nor  visual 
learning,  nor  graphic  design,  nor  aesthetics  of  art  and  so  on,  some  key  leaders  were  greatly  limited  in  their  ability  to 
imagine  the  potential  of  the  visual  messages  within  each  product. 

Certain  weaknesses  within  each  product,  both  in  terms  of  adding  time  to  the  productior  and  in  the  final  visual  message, 
were  the  direct  result  of  decisions  made  by  project  leaders  who  did  not  understand  how  to  take  advantage  of  the  medium  to 
enhance  learning,  tried  to  save  time  by  skipping  steps  in  the  design  phase,  and  placed  too  much  responsibility  on 
programmers  to  provide  design  solutions  toward  the  end  of  the  development  cycle.  The  complexity  of  reasons  for  these 
weaknesses  are  beyond  the  scope  of  this  paper,  but  the  resulting  compromise  in  potential  learning  is  most  evident  in 
certain  mismatches  between  video  imagery  or  graphics  and  the  content  or  tone  of  the  script. 


There  is  no  doubt  that  the  single  most  important  factor  in  the  successful  instructional  design  and  imagery  used  was  the 
result  of  teamwork  that  drew  on  the  talents  of  a variety  of  people.  Communication  among  the  design  team  was  essential. 

Successful  visualization  within  each  of  the  products  was  the  direct  result  of  being  able  to  work  with  a talented  artist  who 
was  able  to  shape  the  ideas  into  visual  form  that  carried  meaning  into  the  instructional  process.  In  order  to  do  that,  the 
artists  needed  to  be  emersed  in  the  design  process  from  the  beginning.  They  turned  ideas  into  images  that  communicated 
visually.  Their  choices  of  shapes,  colors,  representations,  animations,  morphing,  and  transitions  had  a major  influence 
in  each  of  the  final  products.  The  visual  elements  that  seem  most  important  are  those  that  comprise  the  basic  structure 
of  the  image,  add  meaning  to  the  message,  and  employ  the  power  of  visual  impact  when  appropriate  (Knupfer,  1994b). 

The  Structure  of  the  Image 

The  structure  of  the  image  first  considers  the  visual  as  a whole,  then  its  components  and  the  elements  of  good  screen 
design.  In  order  for  the  various  image  components  to  work  together,  elements  of  well-planned  screen  design  must  weave 
the  components  together  in  an  aesthetically  pleasing  and  understandable  format.  Thus,  one  must  consider  both  the 
separate  image  components  and  the  screen  design  as  a whole. 
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The  Image 

The  image  contains  a mix  of  components  that  can  work  together  or  separately  to  modify  the  intended  message.  Text, 
color,  graphics,  animation,  the  multimedia  mix,  size,  special  effects,  and  the  dynamic  nature  of  the  components  all  will 
influence  the  message  delivery  and  resulting  interpretation. 

Text  In  the  late  1980s,  text  dominated  cQmputer-based  instruction  (Soulier,  1988).  Improved  technology  and 
attention  to  visual  design  offer  the  potential  for  more  variety  within  the  imagery,  and  many  courseware  designers  have 
incorporated  more  graphics.  Yet  it  appears  that  much  of  the  information  available  from  computer  courseware  and  from 
internet  resources,  does  not  take  advantage  of  this  capability  and  seems  to  use  text-intensive  screens. 

Analysis  of  the  social  worker’s  courseware  revealed  a strong  effort  at  visual  design.  Graphics  and  video  were  incorporated 
throughout  the  courseware,  yet  the  majority  of  information  was  contained  within  the  text,  with  the  imagery  used  for 
emphasis,  reinforcement,  or  aesthetic  value.  Analysis  of  the  teacher’s  courseware  revealed  that  graphics  were  used  to 
carry  the  message  in  certain  places  and  to  support  the  textual  message  in  other  places.  Both  the  social  worker’s  and 
teacher’s  courseware  depended  heavily  on  the  audio  script  to  support  video  imagery.  In  addition,  the  social  work  material 
offered  the  learners  the  ability  to  view  the  text  of  any  audio  script  in  addition  to  hearing  it.  None  of  the  learners  utilized 
this  feature,  yet  it  was  left  in  the  product  to  accommodate  learners  who  might  be  hearing  impaired  or  use  the  material  in 
an  office  with  environmental  noises  that  might  interfere  with  their  ability  to  hear  the  audio.  Both  products  incorporated 
volume  control. 

When  graphics  alone  are  used  to  carry  the  message,  the  graphics  must  be  concise  enough  so  that  learners  are  able  to 
understand  the  message.  Likewise,  when  large  amounts  of  text  are  presented,  it  is  important  to  design  text  displays  that 
communicate  clearly  to  the  reader.  With  appropriate  fonts  and  spacing,  computer  screens  can  work  very  well  for 
presenting  limited  amounts  of  text.  Large  amounts  of  text  work  better  in  printed  form  (Soulier,  1988)  since  they  induce 
eye  fatigue  (Hathaway,  1984;  Mourant,  Lakshmanan,  & Chantadisai,  1981)  and  are  likely  to  be  forgotten  when  presented 
on  the  screen  (Wager  & Gagne,  1988). 

In  addition  to  the  amount  of  text,  its  density,  along  with  typographic  cueing  and  the  mix  of  upper-case  and  lower-case 
characters  can  affect  legibility  (Hartley,  1987;  Hathaway,  1984;  Morrison,  Ross,  & O'Dell,  1988;  Ross,  Morrison,  & 
O'Dell,  1988).  Variations  in  font  type,  size,  and  density  along  with  direction  and  screen  placement  can  add  meaning  to 
the  text  image.  With  the  exception  of  selected  levels  of  titles  and  headings,  or  special  effects,  the  text  should  contain  a 
mix  of  upper  and  lower  case  letters. 

The  teacher’s  product  did  not  contain  titling  and  depended  upon  the  graphics  of  the  user  interface  to  act  as  a map  for  the 
learner.  For  example,  starting  with  a hallway  lined  with  various  doors,  the  learner  would  click  on  a doorway  of  a specific 
room  and  by  moved  into  that  room.  From  that  point,  the  learner  used  objects  within  that  room  until  the  hallway  pass 
was  chosen  to  exit  that  room  and  return  to  the  hallway.  The  social  worker’s  material  began  by  using  a single  level  of 
titling,  but  changed  to  a more  precise  title  plus  subtitle  system  after  formative  evaluation  revealed  confusion. 

No  matter  what  the  screen  design,  legible  text  requires  an  appropriate  font  that  is  properly  spaced.  There  is  disagreement 
about  the  appropriateness  of  serif  or  sans  serif  style  fonts.  Some  authors  claim  that  fonts  with  serifs,  as  opposed  to 
block-style  lettering,  are  a better  choice  for  computer  screens  (Soulier,  1988).  Yet  others  believe  that  sans  serif  fonts 
with  proportional  spacing  provide  a cleaner  effect  that  is  easier  to  read  than  their  seriffed  counterparts  (Gibson  & Mayta, 
1992;  Kemp  & Dayton,  1985).  Fonts  with  small  serifs  can  add  interest  to  the  display,  while  elaborately- seriffed  fonts  are 
difficult  to  read,  especially  if  the  image  is  to  be  viewed  on  television  or  projected  in  a large  room. 

Text  legibility  also  depends  upon  the  point  size  of  the  font.  The  point  size  of  text  on  a computer  screen  can  range  from 
12  to  over  100  points  per  inch  and  remain  legible.  However  one  should  consider  the  intended  usage  when  selecting  point 
size.  Images  that  are  likely  to  be  projected  for  large  audience  reading  should  not  use  a point  size  smaller  than  26  (Gibson 
& Mayta,  1992)  and  images  that  are  not  likely  to  be  projected  or  broadcast  will  seldom  require  an  extremely  large  font. 
Also  consider  the  user;  when  designing  screens  for  young  children  or  people  with  visual  impairments,  use  a larger  font. 

In  addition  to  style  and  size  of  the  font,  the  weight  of  the  typeface,  line  length,  phrasing,  and  spacing  between  lines  of 
text  affect  the  legibility  of  computer  screens  (Jonassen,  1988;  Morrison,  Ross  & O’Dell,  1988).  The  weight  of  a font 


can  vary  from  light,  narrow,  fine  lines  to  heavy,  broad,  bold  lines.  A medium  to  bold  weight  is  very  good,  depending 
upon  the  mix  of  elements  on  the  screen.  Gibson  and  Mayta  (1992)  recommend  that  bold  typeface  be  used  throughout  all 
computerized  screen  images  that  are  intended  for  broadcast  so  that  the  text  shows  up  against  the  graphics.  At  the  least 
bold  fonts  should  be  used  for  all  titles  and  headings  as  well  as  for  particular  words  that  need  emphasis  in  a projected 
image.  Bold  fonts  are  not  necessary  if  the  image  is  to  be  viewed  on  a single  computer  screen.  Medium  weight  fonts  are 
quite  legible  if  the  color  mix  and  font  style  are  compatible.  If  using  a small  font  to  preset  a large  amount  of  text  on  an 
individual  screen,  a lighter  weight  font  will  be  more  legible. 

Drop  shadows  behind  the  text  characters  can  add  legibility  if  used  carefully,  but  they  can  also  interfere  with  legibility. 
Therefore  it  is  important  to  check  legibility  prior  to  applying  drop  shadows  throughout  the  courseware. 

The  design  of  hypermedia  resources  must  consider  the  effects  of  mixing  text  in  a way  that  follows  known  layout  practices 
to  aid  learning  and  retention,  while  attending  to  individual  preferences,  habits,  and  usage  situations.  All  of  this  must 
consider  mixing  media  in  a way  that  allows  learners  to  recheck  information  that  they  do  not  commit  to  memory. 

Text  that  is  designed  as  part  of  the  visual  imagery  can  add  impact  to  the  on-screen  presentation.  This  visual  text  can  be 
stagnant  or  dynamic  and  has  the  ability  to  add  meaning  to  the  message.  Text  can  employ  hotlinks  to  more  information 
and  color-coding  to  suggest  meaning  and  serve  as  an  aid  to  navigation.  Although  it  is  tempting  to  use  flashing  text  as  a 
way  of  getting  attention  or  using  a fancy  technique,  this  should  not  be  done.  The  flashing  can  interfere  with  the  brain 
functions  of  people  who  are  light  sensitive  or  who  have  a seizure  disorder. 

No  matter  what  exact  techniques  are  used  from  the  range  of  possibilities,  learners  seem  to  be  more  able  to  find  their  way 
around  when  the  user  interface  is  simple  and  consistent  so  learners  know  what  to  expect.  Thus  for  text-dense  sections,  it 
is  better  to  design  screens  that  present  categories  of  information  concisely  with  links  to  other  screens  rather  than 
presenting  long  scrolling  screens  of  text.  If  the  navigational  interface  includes  consistent  return  paths  following  each 
section,  then  the  learners  6an  navigate  with  less  chance  of  getting  lost. 

Color.  Specific  educational  objectives  can  be  enhanced  by  using  color  in  visual  illustrations  (Dwyer,  1978),  but  while 
a few  colors  can  cue  the  learner  about  the  intended  message,  too  many  colors  can  be  confusing.  Color  should  assist  the 
user  in  focusing  on  the  material;  it  should  never  be  a distraction  (Gibson  & Mayta,  1992).  Accordingly,  Hannafin  and 
Peck  (1988)  suggest  using  a bright  color  to  cue  the  learner  for  new  information,  while  presenting  the  remainder  of  the 
information  in  standard  colors  consistent  with  the  rest  of  the  screen.  Soulier  (1988)  recommends  checking  the  program 
on  a monochrome  monitor  and  when  in  doubt  about  legibility,  use  a different  color  or  employ  patterns  as  a backup 
technique  to  aid  those  people  who  are  color  blind. 

Color  can  be  used  as  a visual  cue  to  indicate  hot  words,  type  of  multimedia  activity  or  link,  flow  of  information  such 
introductory  or  summary  sections,  or  as  a navigational  aid  to  highlight  menu  choices,  content  maps,  and  so  on. 

Whenever  possible,  it  is  helpful  to  use  colors  in  a way  that  does  not  conflict  with  outside  materials  that  are  likely  to  be 
incorporated  as  part  of  the  instructional  flow.  For  example,  if  learners  will  search  the  internet  as  part  of  the  instruction, 
then  color  coding  of  hot  links  should  be  used  in  a way  that  is  likely  to  be  employed  in  those  materials  as  well  as  opposed 
to  simply  choosing  your  color  coding  independency  of  that  situation. 

A few  colors  with  good  contrast  values  will  show  up  well  on  both  color  or  monochrome  displays,  but  an  extreme 
contrast  like  stark  white  on  a black  background  will  cause  bleeding  and  illegibility;  it  is  better  to  use  light  gray  to 
achieve  the  desired  effect.  Also  avoid  high  values  of  red  and  orange  because  they  can  bleed  into  the  surrounding  colors. 

In  addition,  certain  colors  that  look  ideal  on  an  individual  computer  screen  tend  to  flare  or  wash  out  when  they  are 
projected  to  a large  screen  or  transmitted  over  a distance.  To  avoid  disappointment  it  is  best  to  experiment  with  a few 
color  combinations  using  the  equipment  that  will  support  the  image  when  it  is  actually  projected.  Complementary 
colors  with  low  saturation  would  have  a good  chance  of  working. 

Graphics  and  Animation . Graphics  add  interest  to  the  screen  by  providing  visual  variety  (Kemp  & Dayton,  1985) 
and  offer  another  opportunity  to  suggest  meaning  to  the  learner.  There  are  many  ways  to  incorporate  graphics  into 
learning,  yet  designers  often  forget  to  determine  the  instructional  reasons  for  using  graphics  prior  to  making  decisions. 
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Five  main  categories  of  instructional  applications  of  graphics  include  cosmetic,  motivational,  attention-gaining, 
presentation,  and  practice  (Rieber,  1994).  Possible  graphic  treatments  range  from  simple  to  complex,  from  small 
monochromatic  embellishments  to  dramatic,  richly-colored,  full-motion  video  images  complete  with  sound  effects.  Prior 
to  deciding  on  uses  of  graphics,  it  is  important  to  consider  the  instructional  meaning  of  the  graphic.  In  the  social  work 
project,  this  guideline  was  extremely  difficult  maintain  because  like  many  others,  key  people  in  authority  positions  had 
been  fooled  into  thinking  that  dramatic  visual  effects  translated  into  dramatic  learning,  regardless  of  instructional  purpose. 

Graphs  that  display  data  depend  upon  the  reader's  thought  processing,  interpretation,  and  comprehension;  to  be  effective 
they  must  consider  the  intended  visual  message  carefully.  Sophisticated  graphic  displays  that  are  designed  to  suggest 
inferences,  generalizations,  and  evaluative  interpretation  can  help  students  interpret  meaning  (Reinking,  1986;  Singer  & 
Donlan,  1980).  High-level  instructional  graphics,  such  as  symbolic,  schematic,  or  figurative  displays  can  be  effective  in 
teaching,  and  the  visualization  of  abstract  ideas  through  figurative  displays  may  very  well  enhance  learning  (Nygard  & 
Ranganathan,  1983).  All  graphics  do  not  require  the  same  level  of  detail  and  clarity,  but  even  simple,  decorative  graphics 
at  the  pictorial  level  have  their  place.  Indeed,  Boyle  (1986)  suggests  a need  for  designers  to  address  cognitive  processes 
by  developing  more  materials  for  graphic  thinkers,  not  just  graphic  readers. 

The  Child  Development  module  contained  a challenge  that  was  nicely  resolved  through  visual  graphics.  Rather  than 
simply  presenting  a growth  chart  as  a line  graph,  the  information  was  displayed  interactively.  Two  figures,  one  girl  and 
one  boy,  were  compared  side  by  side  on  the  screen.  Learners  could  enter  information  about  age,  height,  or  weight  and  see 
visual  representations  that  compared  boys  to  girls  at  key  stages  of  growth.  The  combination  of  interactive  visualization 
seemed  to  help  learners  conceptualize  important  stages  of  physical  development. 

To  assist  with  designing  computer  graphics,  Soulier  (1988)  offers  the  following  guidelines:  keep  illustrations 
appropriate  for  the  audience;  Use  simple  line  drawings  when" possible;  preload  graphics  into  the  program  so  that  they 
appear  quickly  on  the  screen;  use  standard  symbols  and  symbolic  representations;  and  keep  graphics  on  the  same  screen  in 
close  proximity  to  the  corresponding  text  message.  These  few  tips  can  greatly  improve  the  aesthetic  appeal  of  a graphic 
and  promote  clarity  of  the  message.  Even  though  multimedia  supports  elaborate  graphics  with  a host  of  detail  and  many 
colors,  use  care  in  developing  those  graphics  so  that  they  fit  the  need.  Sometimes  it  will  be  important  to  provide  near 
photographic  quality  images,  but  other  times  that  is  completely  unnecessary  and  only  takes  up  memory  while  slowing 
the  program  processing. 

Most  development  software  allows  the  use  of  simple  animation  to  illustrate  a motion,  provide  interest,  or  draw  attention 
to  particular  areas  of  the  screen  (Kemp  and  Dayton,  1985).  Although  it  is  tempting  to  embellish  the  graphic,  it  is 
important  to  use  animations  only  when  appropriate  and  keep  them  short  (Soulier,  1988). 

In  order  for  animations  to  be  effective,  there  must  be  a need  for  visualization.  Animate  only  a few  graphics  that  will  aid 
the  learners’  comprehension  of  the  material  by,  for  example,  gaining  attention,  illustrating  a concept,  or  showing 
direction.  When  used  sparingly  animations  can  be  effective,  but  they  can  become  irritating,  distracting,  or  disruptive  to 
the  thought  process  if  overused  or  left  on  too  long.  Whenever  possible,  allow  the  user  to  interrupt  the  animation. 

Screen  Design 

Good  screen  design  aids  the  learner  by  using  visual  components  to  portray  the  message  in  a way  that  provides  both 
clarification  of  information  and  visual  interpretation.  To  visually  aid  the  learner,  it  is  necessary  to  consider  the  specific 
elements  of  good  screen  design  as  well  as  the  general  screen  layout. 

Elements  of  Good  Screen  Design.  Interesting  screens  are  composed  of  a variety  of  elements  that  work  well 
together.  Many  of  these  elements  are  equally  important  so  they  are  presented  here  in  no  particular  order.  The  goal  of 
good  screen  design  is  to  use  the  various  elements  together  to  compose  a simple,  consistent  design  that  provides  sufficient 
information  while  avoiding  clutter. 

Unlike  printed  material  which  can  be  skimmed  at  will,  the  computer  screen  limits  the  learners’  view  of  the  overall 
content.  Screen  designs  that  are  simple,  straight  forward,  and  consistent  can  help  lead  learners  through  the  'atcrial, 
while  complicated  designs  can  lead  to  frustration.  The  basic  simplicity  of  frame  layout  and  user  options  does  not  restrict 
the  ability  to  add  interest  and  meaning  with  a full  range  of  simple  and  complex  graphics. 
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The  graphic  user  interface  in  the  teacher’s  multimedia  project  presented  a simplified  view  of  a school.  Each  room 
contained  certain  information  that  would  be  familiar  and  suggest  meaning  to  the  learners,  but  extra  elements  were  left 
out.  Possible  problems  with  such  a graphical  user  interface  are  likely  arise  when  the  imagery  or  graphical  metaphor 
presents  an  environment  that  is  unfamiliar  to  the  learners.  Also,  it  is  important  to  establish  patterns  of  linkages  within 
the  graphics  so  that  the  learners  know  what  to  expect.  For  example,  the  images  that  contain  hotlinks  should  be  obvious 
to  the  learners.  They  should  not  have  to  makes  guesses  about  which  images  are  active  and  which  are  not  active. 

Certain  user  options  should  always  be  available.  For  example,  status  lines  at  the  top  or  bottom  of  the  screen  that 
contain  consistent  information  help  the  learner  assess  progress  and  maintain  some  control  over  the  program  direction. 

The  reader  should  control  the  display  rate  when  possible  so  that  there  is  adequate  time  to  read  the  text,  interpret  the 
graphics,  and  consider  the  meaning  of  the  message.  A few  standard  templates  will  aid  programmers  and  learners. 
Programmers  can  easily  update  images  or  text  if  fits  into  a template  without  experiencing  misalignments.  If  designed 
consistently,  learners  know  what  to  expect  from  patterns  of  information  and  program  functionality  within  template  areas. 

Menus  should  be  clear,  concise,  uncluttered,  and  consistent.  Icons  within  menus  can  be  very  helpful  if  the  meaning  of  the 
icon  is  readily  apparent.  The  range  of  possible  choices  in  some  programs  can  lead  to  cluttered  or  excessively  layered 
menus;  pull-down  menus  can  be  a solution.  Highlighting  or  fading  some  menu  choices  will  quickly  give  a visual  cue 
about  which  items  are  currently  available. 

Careful  positioning  of  text  on  the  screen  can  add  to  its  aesthetic  appeal  and  legibility.  Although  centering  can  work  for 
lists,  diagrams,  or  graphic  mixes,  most  text  should  be  left  justified  and  limited  to  65  characters  per  line  or  25  characters 
per  line  for  projected  images  (Knupfer,  1994a).  Partitioned  screens  in  which  text  is  confined  to  specific  areas,  can  work 
very  well. 

Ross  and  Morrison  (1988)  suggest  using  a hierarchical  text  display  that  is  vertical  and  uses  indentations  similar  to  an 
outline.  They  further  recommend  a low-density  text  display  with  reduced  wording  and  sentences  limited  to  one  main  idea. 
Also,  it  is  important  to  use  care  when  splitting  lines  so  that  phrases  remain  complete  Soulier  (1988).  Personal 
preference  varies  concerning  the  spacing  of  text  as  long  as  it  is  legible,  but  do  provide  text  breaks  where  the  content 
allows.  In  the  case  of  a completely  graphic  menu,  the  images  should  be  limited  to  those  that  are  necessary  and  should 
also  suggest  an  obvious  meaning.  There  is  disagreement  about  whether  icons  should  be  labeled,  but  one  compromise  we 
can  suggest  is  to  provide  labels  within  balloon  help.  The  social  worker’s  product  incorporated  icon  labeling  in  that 
manner  so  that  a label  appeared  only  when  the  mouse  passed  directly  over  an  icon. 

In  addition  to  font  size,  the  text  legibility  is  influenced  by  contrast  with  the  background.  Common  considerations  for 
both  computer  screens  and  video  images  suggest  cool,  neutral  background  colors  like  gray  or  blue  instead  of  bright,  very 
light,  or  very  dark  backgrounds.  Tasteful  use  of  enhancements  such  as  outlined,  inversed,  flashing,  or  drop  shadowed  text 
can  add  to  legibility.  The  wide  variety  of  patterns  available  for  backgrounds  can  create  visual  interest,  but  those 
background  patterns  can  also  interfere  with  the  legibility  of  text. 

Special  techniques  for  changing  the  screen  display,  such  as  zooming,  panning,  tilting,  and  wiping  onto  the  screen,  can 
vary  the  viewer’s  perspective  of  the  image.  For  example,  a section  of  the  screen  can  be  enlarged  to  give  a close-up  view 
of  specific  details.  Or  the  image  can  change  from  a long  shot  displayed  in  a small  part  of  the  screen  to  an  extreme  close- 
up  showing  part  of  the  same  image  displayed  in  full  screen  mode.  This  technique  can  give  the  learner  the  sense  of 
moving  in  to  take  a close  look  at  the  image.  Imagine,  for  example,  the  visual  effect  of  looking  at  a long  shot  of  a group 
of  trees  in  a small  box  on  the  screen  and  then  changing  to  a full  screen  display  of  a close-up  shot  of  leaf  on  one  of  those 
trees. 

When  projecting  computer  images  to  a large  screen  system,  overscan  and  underscan  considerations  become  important 
(Knupfer,  1994a).  Overscan  fills  the  screen  beyond  the  edges  so  that  no  blank  space  will  show  around  the  edge  when  the 
image  appears  on  a television  monitor;  computer  graphics  need  to  be  produced  in  overscan  mode  so  that  no  blank  edges 
or  distracting  video  signals  will  show  around  the  edges  during  transmission  (Gibson  & Mayta,  1992).  Underscan  protects 
a blank  area  around  the  screen  edge  so  that  images  don’t  get  cut  off  during  the  transmission  process;  important 
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information  should  be  placed  within  a safe  area,  usually  the  middle  two-thirds  of  the  screen  (Kemp,  1980).  This  is 
especially  important  when  determining  proper  placement  of  navigational  aids  at  the  screen  edges. 

Both  the  social  worker’s  and  teacher’s  multimedia  products  in  this  study,  were  designed  with  for  use  by  individual 
learners,  but  the  products  also  needed  to  function  for  large  group  presentation  in  certain  situations,  so  issues  of  text  size, 
overscan  and  underscan  were  important.  A second  common  problem  with  computer-produced  screens  is  changes  in  color 
that  emerge  as  the  image  passes  through  different  equipment.  For  example,  color  selection  might  not  hold  up  during 
projection,  colors  might  appear  different  or  even  washed  out  on  different  screens.  Even  if  it  looks  wonderful  on  the 
production  machine,  always  check  your  images  on  equipment  and  situations  available  to  the  end  users. 

Screen  Layout.  The  elements  of  good  screen  design  work  together  to  build  a cohesive  screen  layout.  The  computer 
screen  layout  should  never  be  visualized  as  a printed  page  filled  with  text,  but  guidelines  similar  to  those  offered  for 
desktop  published  materials  can  be  helpful.  These  include  balancing  text  with  white  space,  improving  the  aesthetics  of 
the  page,  and  positioning  graphics  as  the  dominant  visual  element  (Parker,  1987).  Designing  the  display  with  attention 
to  legibility,  the  purpose  of  the  particular  frame,  and  consistent  protocol,  can  result  in  visually  interesting  computer 
screens.  Avoid  cluttering  the  screen  with  too  many  images;  provide  print  copies  of  complicated  images  that  are 
important  to  remember.  The  social  worker’s  project  produced  a workbook  with  text  intensive  and  supporting  materials 
that  were  best  offered  in  print.  Support  material  included  pages  about  child  development  that  could  be  Xeroxed  and  set 
home  with  parents.  These  worksheets  would  be  especially  valuable  to  parents  who  are  tracking  patterns  in  children 
suspected  of  having  delayed  physical,  mental,  or  emotional  development. 

Like  silence  within  oral  communication,  empty  spaces  can  be  used  to  advantage  on  the  computer  screen.  For  example 
the  screen  can  be  used  to  organize  or  highlight  information,  to  draw  attention  to  particular  parts  of  the  frame.  The  mix  of 
graphics  and  text  can  provide  a visual  cue;  so  can  boxing  and  grouping  of  information.  Partitions,  borders,  standard 
icons,  and  consistent  placement  of  common  elements  will  visually  aid  the  reader. 

While  partitions  and  borders  can  draw  attention  to  an  area,  artistic  sense  can  still  flourish.  Double  borders  or  pseudo 
bordered  areas  can  create  an  interesting  effect,  but  use  care  when  partitioning  screens.  Screens  areas  can  help  learners 
know  where  to  look  for  certain  types  of  information,  but  excessive  use  of  boxes  and  boarders  can  also  disrupt  the  flow  of 
information  and  signal  novice  design  skills. 

Headings  are  often  centered  and  bold,  or  sometimes  even  boxed.  Although  this  is  not  necessary  for  nonbroadcast 
multimedia,  headings  should  at  least  be  used  consistently.  For  long  or  complicated  sequences  of  instruction,  subheadings 
can  be  used  that  include  numbers  or  roman  numerals  to  aid  the  reader  in  visually  following  the  general  flow  of 
information.  As  a general  rule,  information  should  flow  from  the  top,  left  part  of  the  screen  to  the  bottom,  right  part  of 
the  screen  because  that  is  the  way  people  in  our  culture  read.  Variations  in  standard  layout  can  work  if  they  are  with 
purpose  and  fit  the  situation.  Challenges  in  this  area  are  likely  to  arise  when  incorporating  information  from  outside 
sources  or  designing  information  for  a wide  variety  of  users,  in  a multicultural  or  world-wide  sense.  The  projects  in  this 
study  applied  limited  resources  to  specific  and  known  audiences,  so  this  did  not  become  an  issue  beyond  the  normal 
attention  to  gender  and  culture  within  the  language,  imagery,  and  examples  employed. 

Good  layout  technique  depends  upon  an  understanding  that  not  all  computer  frames  are  alike.  Hannafin  and  Peck  (1988) 
address  transitional,  instructional,  and  question  frames.  Transitional  frames  are  used  to  tie  together  the  different  parts  of 
the  a computerized  lesson:  they  provide  an  orientation  to  the  beginning  of,  and  various  sections  within,  the  program; 
they  serve  as  bridges  between  various  topics  or  sections;  they  provide  feedback,  directions,  and  instructions;  and 
periodically,  they  present  a progress  report  to  let  the  learner  gauge  success.  Instructional  frames  present  basic 
information  to  the  learner:  these  frames  can  alert  the  student  to  a need  for  prerequisite  information;  provide  links  between 
relationships  from  past  and  current  learning;  and  provide  definitions,  examples,  and  rules.  Question  or  criterion  frames 
solicit  input  from  the  student  to  help  individualize  the  instruction;  these  frames  usually  are  based  upon  a true  or  false,  yes 
or  no,  multiple  choice,  completion  or  short  answer,  or  a constructed  response  which  is  considered  to  be  a more  open- 
ended  answer. 

There  are  also  variations  to  the  general  type  of  frames.  For  example,  sometimes  a frame  contains  both  instruction  and  a 
question.  Copy  frames,  prompt  frames,  hint  frames,  and  interlaced  frames  are  some  types  of  variations  (Hannafin  & 
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Peck,  1988).  Copy  frames  provide  information  and  a question  about  that  information  in  the  same  screen.  This  type  of 
format  can  be  helpful  in  directing  student  attention,  emphasizing  important  points,  and  for  assuring  a high  degree  of 
success  for  particular  students.  But  because  they  are  so  obvious,  copy  frames  are  considered  very  elementary  and  need  to 
be  used  sparingly. 

Prompt  frames  direct  the  learner  to  supply  input;  these  can  be  used  effectively  for  questions  as  well  as  instructional 
screens.  Hint  frames  are  usually  provided  after  a student  has  failed  to  enter  an  expected  response;  they  offer  guidance  but 
do  not  supply  the  correct  response.  Interlaced  frames  are  hybrids  which  combine  various  components  from  the  standard 
frame  types;  they  might  include  instruction,  question,  and  feedback  all  on  the  same  screen.  This  design  can  appear 
cluttered  if  not  presented  carefully  but  it  has  the  advantage  of  allowing  the  student  to  visually  examine  and  compare  the 
question  and  feedback. 

Each  type  of  frame  depends  upon  grouping  of  information  in  a way  that  visually  aids  the  reader.  To  make  optimal  use  of 
visual  cues,  it  is  helpful  to  design  standard  protocol  for  each  type  of  frame  and  use  it  consistently  throughout  the 
program.  Whatever  protocol  is  chosen  will  need  to  comply  with  the  overall  program  design.  Programs  that  vary  the 
screen  location  of  pertinent  information  or  procedures  used  to  advance  throughout  the  program  can  be  confusing  and 
frustrating  (Mackey  and  Slesnick,  1982).  Although  standard  protocol  is  necessary  (Apple,  1992;  Heines,  1984;  Jonassen, 
1991;  Lentz,  1985;  Simpson,  1984)  designers  can  provide  artistic  variation  to  other  parts  of  the  screen  to  suggest 
meaning. 


The  Meaning  of  the  Image 

Information  becomes  valuable  as  it  takes  on  meaning  for  an  individual.  Since  visuals  are  meant  to  aid  in  the  discovery  of 
meaning,  it  follows  that  well-designed  visuals  will  help  students  interpret  the  meaning.  Computer  images  vary  widely  in 
potential  design  and  usage.  Images  can  be  static  or  dynamic,  concrete  or  abstract,  and  they  can  change  as  a result  of  user 
interaction  with  the  program.  A learner's  interpretation  of  the  image  can  be  affected  by  the  text,  type  of  graphic,  and 
layout  employed. 

Layout  must  consider  the  principles  of  perceptual  organization,  which  include  similarity,  proximity,  continuity,  and 
closure  (Bloomer,  1976).  These  four  processes,  by  which  the  mind  organizes  meaning,  depend  on  how  physically  close 
the  objects  are,  how  similar  they  are,  whether  there  is  a continuous  line  to  guide  the  eye,  and  whether  the  minimal 
amount  of  information  is  present  that  is  necessary  to  obtain  meaning  or  closure.  Comprehension  is  directly  affected  by 
the  way  the  mind  organizes  meaning  from  the  placement  of  graphics  and  text  (Norman,  1993)  and  in  the  process  of 
linking  information. 


The  Power  of  the  Image 

Computer  graphics  and  appropriate  screen  displays  can  lend  power  to  communications  by  adding  vivid  imagery  to  the 
text.  Visual  images  can  aid  message  interpretation  and  enhance  learning.  They  can  also  add  power  to  the  message  by 
providing  an  emotional  element  that  is  beyond  that  of  other  communication  strategies.  Realism  can  be  enhanced  by 
providing  a graphic  component.  Images  can  represent  realistic  data  ranging  from  simple  sketches  or  graphs  to  intricate 
displays  or  vividly  emotional  scenes.  Virtual  reality  can  even  conjure  up  imaginary  situations  through  artificial  imagery. 

Brenda  Laurel  compares  computer  programs  to  theatrical  plays  and  describes  several  similarities  between  the  two  (1993). 
Although  computer  programs  can  allow  different  paths  to  completion,  they  are  similar  to  plays  in  that  they  should  have  a 
beginning,  middle,  and  end.  In  addition,  both  can  provoke  emotions. 

Emotional  impact  can  be  added  to  hypermedia  by  using  graphics  that  evoke  feelings  or  encourage  learners  to  imagine 
certain  situations.  Size,  color,  shape,  dynamic  images,  and  special  effects  can  be  employed  to  evoke  emotions,  but  it  is 
wise  to  use  a conservative  approach  and  not  overdue  these  techniques. 

Hypermedia  can  display  still  or  dynamic  photos  of  real  or  imaginary  events.  These  events  and  their  results  can  combine 
with  text  and  audio  segments  to  provide  a sense  of  realism  to  the  user  that  otherwise  would  not  be  possible.  The  Child 
Development  product  evoked  emotional  impact  by  showing  still  and  motion  images  of  children  in  real  situations  of 
school,  play,  homes,  and  hospital  settings.  The  teacher’s  product  showed  real  classroom  scenarios.  Along  with  the 
understanding  provided  by  the  realism  of  these  events,  comes  the  potential  to  stir  emotions  for  various  reasons. 
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What  are  the  proper  uses  of  visual  metaphors? 

Like  verbal  metaphors,  visual  metaphors  can  help  us  to  understand  an  unfamiliar  concept.  Metaphoric  graphics  (Soulier, 
1988)  may  be  used  to  clarify  a meaning  within  the  computerized  message,  or  they  can  guide  the  user  through  the 
mechanics  or  functionality  of  using  the  software.  For  example,  standard  male  and  female  icons  represent  content  in  the 
growth  chart  portion  of  the  Child  Development  product  while  an  exit  sign  icon  symbolizes  the  functionality  of  both 
products.  Both  products  in  this  study  used  a typical  “VCR”  control  panel  to  manipulate  viewing  of  the  video  clips. 

Metaphors  are  only  as  good  as  a situation  allows.  Metaphors  can  work  well  in  a variety  of  situations,  but  in  all  cases 
must  provide  an  appropriate  match  between  the  topic,  the  learners,  and  the  way  the  metaphor  is  applied  to  the  design.  If 
learners  cannot  relate  to  the  metaphor,  then  the  instructional  message  will  be  lost.  But  if  they  can  relate  to  the  metaphor, 
then  the  instructional  message  can  be  enhanced. 

Metaphors  are  used  to  some  degree  in  both  the  social  work  and  the  teaching  multimedia  projects,  but  they  are  applied 
with  differing  intensities.  The  social  work  project  incorporates  a variety  of  limited  metaphors  into  the  modules,  but  does 
not  place  heavy  emphasis  on  does  not  carry  any  one,  single  metaphor  throughout  the  entire  series  other  than  in  the 
overall  functionality  of  the  user  interface.  For  example  the  legal  module  was  developed  by  different  design  team  members 
and  contains  different  metaphors  than  the  stress  module,  yet  the  general  functionality  of  the  user  interface  remains 
consistent. 

The  metaphor  within  the  teacher  multimedia  worked  well  with  the  student  teachers  it  was  intended  to  be  used  with,  but 
other  learning  audiences  have  reacted  with  confusion  and  frustration  . Objections  to  the  completely  graphic  interface 
arose  because  some  learners  could  not  relate  to  the  school  scenario,  needed  more  preliminary  interpretation  of  the 
graphical  user  interface,  or  preferred  environments  that  were  more  text-based.  General  comments  about  desiring  more  text 
indicated  some  potential  for  confusion  with  interpreting  the  school  metaphor  as  well  as  unclear  functionality  of  each 
graphic.  Learner  who  are  willing  to  explore  will  probably  feel  more  comfortable  in  this  type  of  environment. 

What  are  the  formidable  questions  inherent  in  the  presentation  of  the  content  and  images? 

How  many  links  per  screen  are  appropriate?  The  number  of  text  links  per  screen  can  become  quite  dense,  depending  upon 
the  situation  at  hand.  When  there  is  information  that  is  critical  to  the  instruction,  designers  need  to  weigh  the 
consequences  of  allowing  learners  to  explore  branches  of  that  information  in-depth  or  finding  extraneous  information 
versus  concentrating  on  a limited  amount  of  critical  information..  We  found  that  time  and  concentration  efforts  were 
limited  in  both  the  social  work  and  teacher  subjects  who  used  these  products,  so  they  choose  to  get  through  the 
instruction  in  the  quickest  way  possible,  ignoring  many  links. 

The  number  of  links  per  screen  certainly  will  depend  upon  the  purpose  of  the  instruction,  the  audience  involved,  time 
available,  consequences  of  learning  more  or  less  information  about  the  topic,  and  whether  the  links  confound  or  enhance 
the  potential  learning.  It  appears  that  a screen  can  have  numerous  links  as  long  as  there  is  an  obvious  way  to  return  that 
screen,  to  exit  a link  that  is  erroneously  chosen,  to  indicate  which  paths  have  already  been  taken,  and  to  aesthetically 
work  the  links  into  the  screen  without  adding  clutter  that  distracts  from  the  main  flow  of  information. 

The  balance  of  images  per  screen  will  not  be  determined  by  a precise  definition,  but  will  depend  upon  the  purpose  of  the 
imagery,  the  balance  of  the  screen  design,  and  the  functionality  of  the  images.  It  is  perhaps  better  to  design  one  complex 
but  integrated  set  of  images  as  opposed  to  independent,  cluttered,  or  conflicting  images.  In  addition,  the  meaning  of  the 
images  is  important.  All  images  should  fulfill  a specific  purpose.  That  purpose  might  be  to  get  attention,  enhance 
meaning,  make  a powerful  statement,  add  variety,  and  so  on,  but  it  clearly  should  fiot  be  simply  to  fill  a blank  space  on 
the  screen. 

The  length  of  the  video  vignettes  was  a topic  of  debate  with  both  of  the  projects  at  hand.  While  the  vignettes  need  to  be 
long  enough  to  provide  meaningful  scenarios  that  portray  information  clearly,  they  need  to  be  short  enough  to  allow  user 
interaction,  practice,  and  feedback  at  appropriate  places.  Large  amounts  of  video  information  can  be  incorporated  as 
smaller  video  clips  as  long  as  there  is  a common  thread  that  laces  the  information  together  and  promotes  interaction  that 
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helps  learners  make  the  cognitive  link  between  pieces.  If  video  is  shown  in  smaller  clips,  the  challenge  becomes  one  of 
determining  proper  entry  and  exit  solutions.  Transitions  become  very  important  yet  need  to  accommodate  learners  in  a 
hypermediated  context,  not  in  a linear  sense.  Novices  on  our  projects  had  difficulty  designing  a product  with  a 
beginning,  middle,  and  end  that  is  was  not  linear. 

When  does  design  supersede  content? 

Some  people  might  say  never , but  there  are  times  when  the  design  needs  to  supersede  content.  Courseware  must  show 
consistency  within  the  user  interface,  so  once  a design  standard  is  decided  upon  it  must  be  maintained  in  a similar  manner 
throughout  the  instruction.  In  addition,  designers  need  to  be  aware  of  standards  and  common  practices  that  have  been 
incorporated  into  other  software  that  the  learners  have  used  in  the  past.  If  learners  expect  the  software  to  function  in 
specific  ways,  then  it  is  best  to  incorporate  standard  practices  when  possible  so  that  learners  can  concentrate  on  the 
content  and  not  be  confused  by  inconsistencies  of  functionality. 

In  a case  where  the  instruction  is  designed  as  a series  of  modules,  learners  expect  one  module  to  function  in  a similar 
manner  as  another.  Like  a series  of  books  in  a set,  learners  expect  certain  consistencies  within  the  look,  feel,  tone, 
instructional  approach,  and  functionality  of  the  courseware. 

When  content  can  be  delivered  in  a variety  of  ways,  design  can  determine  final  decisions  about  how  content  is  delivered 
and  how  much  detail  is  appropriate.  Because  hypertext  allows  hotlinks  to  be  made,  designers  are  not  pressed  to  include 
all  details  about  a subject  within  the  main  part  of  the  courseware,  but  can  use  links  to  outside  resources  as  necessary, 
thus  enabling  learners  to  choose  more  or  less  detail,  depending  on  how  much  information  they  need  about  specific  topics 
and  how  much  time  they  can  spend  at  the  given  moment. 

Screen  design  enters  into  consideration  as  well.  Good  screen  design  allows  adequate  white  space  so  that  the  eye  has  a 
resting  place  and  incorporates  imagery  in  a meaningful  way.  Rather  than  filling  the  screen  with  text,  designers  often  need 
to  reword  and  abbreviate  text  so  that  it  fits  well  within  the  visual  design  of  the  screen  and  in  so  doing,  need  to  consider 
grouping  of  words,  phrases,  and  lists  in  the  best  way  to  enhance  the  mental  processes.  Substituting  imagery  for  text  can 
add  power  to  the  message. 

Product  design  should  to  meet  the  needs  of  the  intended  audience.  Products  need  to  fit  within  the  intended  learning 
environment  in  numerous  ways  and  in  so  doing,  consider  the  hardware  available,  the  physical  work  environment,  and  the 
emotional  state  of  the  learners.  The  best  products  will  fall  short  of  intended  results  if  people  need  to  struggle  with 
hardware,  can’t  hear  the  audio,  can’t  fit  instructional  sequences  into  available  timeframes,  and  so  on. 

The  audience  might  expect  media  to  contain  special  effects  like  they  see  at  the  movies  or  on  television.  The  maturity  of 
the  audience  could  very  well  be  a factor  in  this  regard  and  should  be  considered.  It  will  be  important  to  use  special  effects 
wisely  so  that  they  add  to  the  purpose  of  the  instruction. 

What  is  the  importance  of  the  instructional  designer’s  familiarity  with  new  models  of  instruction 
and  learning  theory? 

Design  team  members  debated  about  whether  the  same  instructional  approach  could  be  used  for  everyone  who  will  use  the 
courseware.  If  not,  then  how  could  the  designers  accommodated  different  learning  styles  within  the  courseware? 

Like  other  types  of  instruction,  hypermediated  instruction  can  provide  examples,  practice  exercises,  and  feedback  that  will 
use  a variety  of  instructional  techniques  and  appeal  to  different  styles  of  learning.  As  hypermedia  becomes  more 
sophisticated  and  gains  capability  of  incorporating  outside  resources,  designers  can  leave  more  discretion  to  user 
preferences.  It  remains  a challenge  to  design  for  the  ty|>e  of  cognitive  processing  allowed  by  hypertext  while  including 
some  sound  design  practice  from  standard  computer-based  instruction.  Even  though  hypermedia  links  and  standard 
computer-based  instruction  seem  to  approach  design  in  opposite  ways,  cognitive  interpretations  of  instructional  design 
reveal  many  similarities  (Jonassen,  1991). 
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It  will  be  important  that  designers  break  free  of  behaviorist  models  to  the  extent  that  the  courseware  does  not  rely  so 
much  on  that  approach.  Hypermedia  allows  a much  more  creative  approach  to  instruction  than  the  drill  and  practice  style 
of  learning,  so  designers  can  incorporate  more  constructivist  approaches  to  courseware  design  and  student  evaluation. 

Designers  can  do  such  things  as  provide  more  choices  within  the  user  interface.  Those  choices  could  allow  learners  to 
move  in  different  paths  through  the  material  and  incorporate  more  oi  less  information  into  the  lesson  as  needed.  In 
addition,  learners  should  not  be  forced  to  listen  to  audio  or  to  read  text  verbatim  at  all  times,  but  should  be  allowed 
choices  to  hear,  read,  or  hear  while  reading  text  in  specific  situations  within  the  learning.  Thus  learners  can  choose  to 
receive  information  in  a way  that  is  most  comfortable  at  a given  time. 

Expectations  for  the  potential  of  hypermedia  are  high,  yet  learners  in  both  the  social  worker’s  and  teacher’s  products 
experienced  a certain  level  of  disorientation  and  they  tended  not  to  use  fancy  features  and  graphic  links.  This  raises 
questions  about  the  need  to  prepare  learners  to  better  use  hypermedia  and  to  better  read  visual  imagery. 

What  theoretical  elements  are  appropriate  for  the  graphical-user  interface? 

It  is  important  to  match  the  instructional  design  to  the  situation  at  hand.  This  means  considering  the  goals  of  the 
instruction,  the  needs  of  the  learners,  the  environment,  and  so  on.  While  doing  so,  designers  must  ask  whether  situated- 
learning  metaphors  are  appropriate  to  the  audience  and  the  content.  If  so,  what  situations  are  suggested  or  accommodated? 

Another  question  that  is  important  to  answer  in  the  design  phase  is  how  the  learning  is  expected  to  be  constructed.  What 
design  elements  related  to  learning  theory  should  be  evident  within  the  courseware?  Is  there  evidence  of  learner  control 
and  autonomy,  or  is  the  program  designed  in  a more  behavioristic,  directed  structure? 

How  does  guided  learning  fit  into  the  courseware?  Is  learning  guided  in  a systematic  way,  a tightly  controlled  way,  not  at 
all,  or  something  in  between?  What  evidence  is  there  of  motivating  learners  and  attending  to  various  learning  theories? 
Does  there  seem  to  be  a match  between  the  courseware’s  purpose,  the  audience,  skill  level,  and  instructional  approach? 
How  does  the  system  work  together  as  a whole? 

The  new  hypermediated  courseware  could  incorporate  information  from  outside  sources  that  the  designer  cannot  determine 
or  control.  For  example,  learners  could  potentially  go  to  certain  pages  in  the  World  Wide  Web  (WWW)  to  get 
information.  The  problem  here  for  instructional  designers  will  be  the  dynamic  nature  of  such  material;  it  could  be  here 
today  and  gone  tomorrow,  or  changed  tomorrow.  If  WWW  resources  are  incorporated,  how  will  it  be  managed? 

As  more  sophisticated  graphics  are  incorporated,  it  will  be  important  to  determine  how  to  manage  the  complexity  created 
by  the  imagery.  How  much  importance  can  be  placed  on  a learner’s  ability  to  read  visual  imagery?  What  things  can  we 
do  to  enhance  visual  literacy?  How  will  windowing  inhibit  or  enhance  learning?  Will  windowing  serve  as  a way  of 
separating  information  visually  or  will  it  interfere  with  the  flow  of  thought?  In  what  ways  can  visual  text  and  visual 
graphics  help  learners  maintain  a sense  of  direction  as  they  move  through  various  links? 

There  are  many  questions  that  lead  good  instructional  design  efforts.  Although  there  are  no  set  answers  lo  all  of  these 
questions,  instructional  designers  can  at  least  address  the  questions  to  determine  what  guidelines  will  lead  the  design 
efforts. 

Are  the  images,  language,  and  content  free  of  cultural  and  gender  bias? 

Media  often  portrays  people  in  stereotypical  roles.  Movies,  television  programs,  advertising,  books,  and  now  even  clip 
art  are  fraught  with  stereotypes  based  upon  cultural  background,  race,  and  gender  (Binns  & Branch,  1995;  Couch,  1995). 
As  courseware  developers  think  about  designing,  scanning,  or  downloading  images  into  the  courseware,  they  should 
consider  what  messages  are  implied  by  the  images. 

Are  the  men  portrayed  in  leadership  positions,  while  the  women  appear  in  subordinate  roles?  Arc  men  portrayed  in  work 
situations  or  using  technology,  while  women  are  portrayed  in  nurturing  roles?  Are  women  portrayed  as  either  older  and 
overly -grandmotherly,  or  young  and  overly-sexy? 
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Are  colors  and  tone  used  to  add  feeling  or  draw  attention  to  an  image  in  a suggestive  way  that  might  bias  the  learners? 
Examples  of  this  are  the  darkening  of  O.  J.  Simpson’s  face  on  the  cover  of  Time  magazine  (1994)  and  the  bright  red 
coloration  applied  to  the  woman  figure’s  nipples  in  an  otherwise  dark  and  muted  image  depicting  a male  and  female 
within  a recent  New  York  Times  (1996)  report  on  cancer.  The  former  suggests  a guilty  verdict  prior  to  the  trial  vhile  the 
latter  continues  the  American  male  fixation  on  women’s  breasts  as  sexual  objects,  even  within  an  article  focused  on 
medical  education. 

Do  sports  analogies  or  competition  appeal  more  to  males  than  females.  Are  cultural  groups  represented  in  true 
proportions  and  in  accurate  situations  with  the  examples,  imagery,  and  prerequisite  knowledge  necessary  for  successful 
completion?  Can  the  courseware  be  used  in  different  cultures  without  bias?  This  question  raises  issues  of  cultural 
sanitation  versus  lack  of  bias;  where  is  the  line  drawn? 


How  do  the  images,  language,  and  content  accurately  represent  the  cultural  and  gender  variables? 

This  question  goes  deeper  into  the  underlying  messages  portrayed  through  the  images,  language,  and  content.  Rather 
than  simply  avoiding  stereotypes,  it  is  important  to  keep  the  messages  in  proper  context  in  terms  of  culture  and  gender. 
For  example,  does  the  design  go  the  extra  step  to  consider  and  include  examples  from  the  cultural  group  that  the 
courseware  will  be  used  with?  Certain  examples  will  make  better  sense  if  the  learners  can  relate  to  them. 

Messages  intended  to  influence  attitudes  toward  using  birth  control  and  planned  parenting,  need  to  consider  the  cultural 
underpinnings  of  such  issues  as  within  the  learning  group.  In  cultures  where  a people’s  status  is  influenced  by  the 
number  or  gender  of  their  of  children,  any  attempt  to  curb  birth  rates  would  need  a very  sensitive  approach.  Likewise, 
portraying  value  systems  of  rural  American  within  inner  city  schools  and  vice  versa,  could  make  the  instruction  fall  flat. 

Icons  and  symbology  can  be  problematic.  While  some  icons  and  symbols  are  interpreted  with  similar  meanings,  others 
will  take  on  different  meanings  within  different  international  audiences.  A dragon,  for  example,  is  thought  of  as  a scary 
and  evil  presence  in  Western  culture,  while  it  is  believed  to  be  powerful  and  even  protective  in  Eastern  cultures. 

both  the  social  worker’s  and  teacher’s  products  were  designed  to  be  used  in  American,  multicultural  situations.  The 
social  work  product  contained  examples  form  both  rural  and  urban  situations  to  allow  more  flexibility  in  meaningful 
usage. 

How  should  formative  and  sununative  evaluation  occur? 

The  main  questions  here  include  the  logistics  of  conducting  formative  and  summative  evaluation  in  a way  that  produces 
valid  results,  questions  about  how  the  visual  elements  actually  enhance  the  learning,  and  determining  what  questions 
regarding  visually  literate  learners  are  appropriate  in  formative  and  summative  evaluations.  The  evaluation  processes  for 
both  products  in  this  study  included  written  questionnaires,  interviews,  pretests  and  posttests,  and  observations.  Yet  the 
evaluations  to  date  raise  addition  questions  about  hypermedia  design. 

Prior  to  designing  the  courseware,  it  will  be  important  to  determine  what  is  important  at  the  evaluation  phases.  What 
will  be  considered  successful  and  what  will  be  considered  problematic?  How  should  the  visual  elements  be  evaluated? 
How  will  designers  determine  the  quality  of  the  use  of  such  things  as  color  and  contrast,  or  whether  the  design  of  the  user 
interface  and  examples  are  appropriate  to  the  content?  Certainly  it  will  be  important  to  consider  the  instructional  goals, 
the  audience,  and  the  situation  in  order  to  answer  these  questions. 

In  addition,  how  are  the  functions  of  the  courseware  evaluated?  What  will  determine  success  when  measuring  links  with 
other  images  and  text?  What  changes  are  expected  in  images  to  enhance  the  learning?  For  example,  will  there  be  visual 
cues  such  as  a color  change  to  indicate  that  a section  has  been  completed  so  that  learners  will  be  able  to  determine  their 
position  within  the  user  interface? 
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What  questions  regarding  visually  literate  learners  are  appropriate  in  formative  and  summative  evaluations?  What  is  the 
best  way  to  determine  learner  understanding  of  the  importance  of  the  image  or  video  to  the  content?  How  can  an 
evaluator  determine  a learner’s  ability  to  “read”  the  image  or  video  in  context  with  the  text  and  other  elements  in  the 
graphical-user  interface?  What  standard  will  determine  the  learners’  acceptance  level  and  comfort  with  the  graphical-user 
interface? 

All  of  these  questions  are  difficult  to  answer  because  they  depend  on  specific  situations  at  hand.  Even  though  hypermedia 
employs  new  techniques  and  holds  the  potential  to  incorporate  dynamic  resources  from  the  internet,  a good  deal  of  design 
and  evaluation  standards  carry  on  from  current  practice.  It  will  be  important  to  note  where  differences  are  important  to 
learning  while  designing  for  the  new  hypermedia  enhancements  and  plan  to  work  within  a framework  that  accommodates 
them  as  well. 

In  addition,  the  hypermedia  links  create  an  element  of  possibility  that  takes  the  learning  process  into  a different 
dimension,  thus  calling  upon  different  ways  of  learning,  using  different  strategies  and  different  ways  of  moving  between 
topics.  Designers  can  no  longer  simply  assume  that  learners  will  move  through  topics  in  any  given  order,  but  must 
allow  them  to  move  around  more  freely.  This  implies  designing  information  in  smaller,  more  independent  sections,  that 
cannot  depend  on  sequence  nor  amount  of  information  covered  to  be  considered  complete.  Designers  will  need  to  find 
ways  to  allow  learners  to  enter  and  exit  software  with  ease  enough  to  accommodate  the  hypermedia,  while  attending  to 
the  completion  of  cognitive  processing  of  chunks  of  information  that  belong  together. 

Understanding  and  emotional  impact  can  be  enhanced  through  powerful  computer  imagery.  The  imagery  can  represent 
real  situations  or  it  can  create  an  artificial  situation  that  appears  to  be  real.  Hypermedia  has  tremendous  potential  in 
terms  of  visual  communications  and  educational  impact,  yet  we  have  just  begun  to  find  ways  to  design  courseware  that 
takes  advantage  of  some  of  that  potential.  Continued  evaluation  is  need  to  examine  learner’s  beliefs  about  their  success 
with  learning  in  hypermediated  environments.  There  is  a need  to  look  not  only  at  the  way  that  instructional  designers 
use  graphical  interfaces  to  produce  hypermediated  courseware,  but  also  the  way  that  students  employ  the  graphics  and 
spatial  organization  to  construct  meaning  or  knowledge. 


REFERENCES 

Andrews,  D.  & Goodson,  L.  (1980).  A comparative  analysis  of  models  of  instructional  design..  Journal  of  Instructional 
Development,  3(4),  2-16. 

Binns,  J.  C.  & Branch,  R.  C.  (1995).  Gender  stereotyped  computer  clip-art  images  as  an  implicit  influence  in 
instructional  message  design.  In  D.  G.  Beauchamp,  R.  A.  Braden,  & R.  E.  Griffin  (Eds.),  Imagery  and  visual 
literacy  (pp.  315-324).  Rochester,  NY:  International  Visual  Literacy  Association. 

Bloomer,  C.M.  (1976).  Principles  of  visual  perception.  New  York:  Reinhold. 

Boyle,  M.W.  (1986).  Hemispheric  laterality  as  a basis  of  learning:  What  we  know  and  don't  know.  In  G.D.  Phye  & T. 
Andre  (Eds.),  Cognitive  classroom  learning:  understanding,  thinking,  and  problem  solving.  San  Diego,  CA: 
Academic  Press,  Inc. 

Clark,  B.I.  (1995).  Understanding  teaching:  An  interactive  multimedia  professional  development  observational  tool  for 
teachers.  Unpublished  dissertation  completed  at  Arizona  State  University,  Tempe,  AZ. 

Couch,  R.  A.  (1995).  Gender  equity  & visual  literacy:  Schools  can  help  change  perceptions.  . In  D.  G.  Beauchamp,  R. 
A.  Braden,  & R.  E.  Griffin  (Eds.),  Imagery  and  visual  literacy  (pp.  105-111).  Rochester,  NY:  International  Visual 
Literacy  Association. 

Dwyer,  F.M.  (1978).  Strategies  for  improving  visual  learning.  State  College,  PA:  Learning  Services. 

Gibson,  R.  & Mayta,  M.  (1992,  August).  Designing  computer-based  instructional  graphics  for  distance  education. 

Paper  presented  at  the  Eighth  Annual  Conference  on  Distance  Teaching  and  Learning.  Madison,  WI. 

Gustafson,  K.  & Powell,  G.  (1991 ).  Survey  of  instructional  models  with  an  annotated  ERIC  bibliography  (2nd  ed). 
ERIC  Document  Reproduction  Service  No.  ED  335  027. 

Hannafin,  M.  & Peck,  M.  (1988).  The  design , development , and  evaluation  of  instructional  software.  New  York,  NY: 
MacMillan  Publishing  Company. 

Hartley,  J.  (1987).  Designing  electronic  text:  The  role  of  print-based  research.  Educational  Communication  and 
Technology , J5(l),  3-17. 


350 


Hathaway,  M.D.  (1984,  January).  Variables  of  computer  screen  display  and  how  they  affect  learning.  Educational 
Technology , 7-11. 

Heines,  J.  (1984).  Screen  design  strategies  for  computer-assisted  instruction.  Bedford,  MA:  Digital  Press. 

Jonassen,  D.H.  (1988).  Instructional  designs  for  microcomputer  courseware . Hillsdale,  NJ:  Lawrence  Erlbaum. 

Jonassen,  D.H.  (1991).  Hypertext  as  instructional  design.  Educational  Technology  Research  and  Development ; 39(1), 
83-92. 

Kemp,  J.  & Dayton,  D.  (1985).  Planning  <£  producing  instructional  media.  Fifth  edition.  New  York,  NY:  Harper  & 
Row,  Publishers. 

Knupfer,  N.  N.  (1994a).  Enhancing  local  and  distance  education  with  computer-produced  visuals.  In  N.  Metallinos  (Ed.) 
Verbo-Visual  literacy:  Understanding  and  applying  new  educational  communication  media  technologies , selected 
readings  from  the  tri-annual  summer  symposium  of  the  International  Visual  Literacy  Association  (TVLA)  in  Delphi, 
Greece,  June  25-29,  1993.  Montreal,  Quebec,  Canada:  3Dmt  Research  and  Information  Center  of  Concordia 
University,  pp.  73-89. 

Knupfer,  N.  N.  (1994b).  Computers  and  visual  learning.  In  D.  M.  Moore  & F.  M.  Dwyer  (Ed.),  Visual  literacy:  A 
spectrum  of  learning.  Englewood  Cliffs,  NJ:  Educational  Technology  Publications,  pp  209-234. 

Knupfer,  N.  N.  (1995).  Developing  hypermediated,  videodisc  training  for  child  welfare  personnel:  Bringing  visually  rich 
training  to  rural  areas.  In  D.  G.  Beauchamp,  R.  A.  Braden,  & R.  E.  Griffin  (Eds.)  Imagery  and  visual  literacy. 
Corsicana,  TX:  The  International  Visual  Literacy  Association,  pp.  351-363. 

Knupfer,  N.  N.,  Barrett,  D.  & Lee,  O.  H.  (1995).  A collaborative  multimedia  development  project  for  rural  training: 
Results  of  a beta  test  and  adjustments  to  design.  In  M.  R.  Simonson  & M.  Anderson  (Eds.)  Proceedings  of  selected 
research  and  development  presentations  the  1995  national  convention  of  the  Association  of  Educational 
Communications  and  Technology  (AECT)  at  Anaheim,  CA,  February.  Ames,  LA:  Research  and  Theory  Division, 
AECT. 

Laurel,  B.  (1993).  Computers  as  theatre.  Reading,  MA:  Addison-Wesley  Publishing  Company. 

Lentz,  R.  (1985).  Designing  computer  screen  displays.  Performance  and  Instruction , 24  (1),  16-17. 

Mackey,  K.  & Slesnick,  T.  (1982).  A style  manual  for  authors  of  software.  Creative  Computing , 8,  110-111. 

Morrison,  G.R.,  Ross,  S.M.,  & O'Dell,  J.K.  (1988).  Text  density  level  as  a design  variable  in  instructional  displays. 
Educational  Communication  and  Technology , 36(1),  103-115. 

Mourant,  S.J.,  Lakshmanan,  R.,  & Chantadisai,  R.  (1981).  Visual  fatigue  and  cathode  ray  tube  display  terminals. 
Human  Factors , 23,  529-540. 

National  Council  of  Teachers  of  Mathematics  (NCTM)  (1991).  Professional  Standards  for  teaching  math  ematics. 

Reston,  VA:  Author. 

New  York  Times  Magazine  (Feb.  11,  1996).  Leading  causes  of  death  from  cancer  in  men  and  women. 

Norman,  D.A.  (1993)..  Things  that  make  us  smart:  Defending  human  attributes  in  the  age  of  the  machine.  Reading, 
MA:  Addison-Wesley  Publishing  Company. 

Nygard,  K.E.,  & Ranganathan,  B.  (1983,  Spring).  A system  for  generating  instructional  computer  graphics.  AEDS 
Journal , /6(3),  177-187. 

Parker,  R.C.  (1987).  The  Aldus  guide  to  basic  design.  Seattle:  Aldus  Corporation. 

Rieber,  L.P.  (1994).  Computers,  graphics,  & learning.  Madison,  WI:  Brown  & Benchmark. 

Ross,  S.M.,  Morrison,  G.R.,  & O’Dell,  J.K.  (1988).  Obtaining  more  out  of  less  text  in  CBI:  Effects  of  varied  text 
density  levels  as  a function  of  learner  characteristics  and  control  strategy.  Educational  Communications  and 
Technology  Journal,  36(3),  131-142. 

Reinking,  D.  (1986).  Integrating  graphic  aids  into  content  area  instruction:  The  graphic  information  lesson.  Journal  of 
Reading,  30(2),  146-151. 

Ross,  S.  & Morrison,  G.  (1988).  Adapting  instruction  to  learner  performance  and  background  variables.  In  D.  H. 
Jonassen  (Ed.)  Instructional  Designs  for  Microcomputer  Courseware  (pp  227-245).  Hillsdale,  NJ:  Lawrence 
Erlbaum  Associates,  Inc.,  Publishers. 

Simpson,  H.  (1984).  A human-factors  style  guide  for  program  display.  In  D.  F.  Walker  and  R.  D.  Hess  (eds.) 
Instructional  Software:  Principles  and  Perspectives  for  Design  and  Use.  Belmont,  CA:  Wadsworth. 

Singer,  H.,  & Donlan,  D.  (1980).  Reading  and  learning  from  text.  Boston:  Little,  Brown. 

Soulier,  J.  S.  (1988).  The  design  and  development  of  computer  based  instruction.  Newton,  MA:  Allyn  and  Bacon,  Inc, 
a Division  of  Simon  and  Schuster. 

Time  Magazine  (1994).  Cover  Illustration 

Tufte,  E.  (1983).  The  Visual  Display  of  Quantitative  Information.  Cheshire,  CN:  Graphics  Press. 


BESTCOPYAVAILABLE 


Wager,  W.  & Gagne,  R.  (1988).  Designing  Computer-Aided  Instruction.  In  David  H.  Jonassen  (Ed.),  Instructional 
Designs  for  Microcomputer  Courseware  (pp  35-60).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates,  Inc.,  Publishers. 


Title: 


Inhibitions  Within  Idea  Generating  Groups:  An  Alternative  Method  of 

Brainstorming 


Author: 


Tim  Kochery  (ABD) 

College  of  Education,  Department  of  Instructional  Systems 

University  of  Minnesota 


Introduction 

Alex  F.  Osborn's  group  brainstorming  treatment  (1953)  remains  the  most  frequently  applied  and  instructed 
procedure  for  the  creative  generation  of  ideas  despite  considerable  evidence  that  demonstrates  its  ineffectiveness.  It  is  a 
remarkable  contradiction  between  a popular  technique  and  a body  of  non-supportive  research  evidence  which  is  based  on 
numerous  replications  (over  20)  of  the  classic  experiment  by  Taylor,  Berry,  and  Block  (1957),  and  which  demonstrates 
that  individuals  working  alone  consistently  produce  nearly  twice  as  many  ideas  as  members  working  in  groups.  Thus, 
group  brainstorming  also  becomes  an  intriguing  example  of  a problem  solving  scenario  where  groups  apparently  do  not 
out-perform  individuals. 

Among  the  explanations  for  inferior  group  performance  is  the  effect  of  "group  inhibition"  which  was  originally 
proposed  by  Dunette,  Campbell  and  Jastaad  (1963).  Various  other  experiments  provide  evidence  that  there  exist  many 
factors  which  can  inhibit  group  performance  and  they  have  been  tied  together  into  a construct  called  "group  effects".  Of 
the  many  "group  effects"  that  have  been  postulated  by  researchers,  six  effects  have  demonstrated  empirical  evidence  of 
their  negative  effect  on  a group's  idea  production:  status  in  the  group,  social  loafing,  task  involvement,  cognitive  inertia, 
self-oriented  needs,  and  communications  apprehension  (Thornburg,  1991). 

Bouchard  (1969)  and  Jablin  (1977)  provide  evidence  that  test  scores  on  the  California  Psychological  Inventory 
(CPI)  and  the  Personal  Report  of  Communication  Apprehension  (PRCA),  consistently  relate  to  individual  performance 
within  groups.  Individuals  with  low  scores  on  these  scales  consistently  evidence  high  productivity  in  groups  and  can  be 
categorized  as  "Low  Communication  Apprehensives".  Conversely,  individuals  with  high  scores  on  the  PRCA 
demonstrate  low  productivity  in  groups  and  can  be  categorized  as  "High  Communication  Apprehensives".  Jablin's  study 
demonstrated  that  "Low  Apprehensives"  (LA)  consistently  produce  more  ideas  in  groups  than  do  "High  Apprehensives" 
(HA),  and  that  HA  individuals  performed  best  when  working  alone.  However,  group  composition  based  on 
communication  apprehension  is  a factor  that  is  frequently  omitted  yet  evidences  a negative  effect  on  overall  group 
productivity,  as  weli  as  individual  learning  and  satisfaction. 

Another  negative  influence  on  the  generation  of  ideas  is  the  effect  of  the  original  brainstorming  treatment  itself, 
which  places  an  emphasis  on  the  group  process.  Diehl  and  Stroebe  (1991)  examined  group  communication  processes  and 
found  that  ’’production  blocking"  is  the  primary  cause  of  low  idea  generation  by  groups.  "Production  blocking"  is  caused 
by  the  normal  group  convention  of  waiting  until  other  members  have  stopped  talking  before  they  can  report  an  idea  that 
has  occurred  to  them.  This  appears  to  block  the  production  of  new  ideas  and  may  be  caused  by  limitations  of  short-term 
memory,  rehearsal  time,  cognitive  distraction  or  cognitive  overload.  Because  these  communication  "blocks"  are  present 
in  any  brainstorming  activity,  or  any  group  discussions,  their  findings  have  lead  them  to  the  emphatic  conclusion  ... 
that  group  sessions  should  not  be  used  to  generate  ideas". 

However,  there  also  exists  evidence  that  the  appropriate  use  of  group  interactions,  as  a social  processing  period 
that  causes  cognitive  arousal,  does  promote  increased  idea  generation.  Studies  by  Dunette  et.  al.(1963)  and  Andre  et 
al.(1979),  both  discovered  that  individual  brainstorming  sessions  became  more  productive  when  they  followed  a group 
brainstorming  session.  These  studies  demonstrated  that  the  group  process  is  a stimulating  catalyst  for  the  subsequent 
individual  activity.  Diehl  and  Stroebe  propose  that  an  individual  session  preceding  the  group  brainstorming  session 
would  allow  time  to  cognitively  rehearse  the  development  of  ideas  and  might  counter  the  effects  of  production  blocking. 
Furthermore,  the  results  of  the  studies  by  Bouchard  and  Jablin  indicate  that  certain  individuals  with  the  personality  trait  of 
communication  apprehension  will  perform  better  when  allowed  to  work  individually. 

The  IGP  (Individual  Orientation-Group  Interactions-Personal  Reflection)  has  been  developed  by  the  author  as  an 
alternative  treatment  for  brainstorming,  which  combines  the  benefits  of  personal  cognition  and  individualized  creative 
thinking  with  the  social  stimulation  of  group  interactions.  The  IGP  is  based  on  an  instructional  design  that  provides  a 
methodology  to  overcome  group  inhibitions,  assure  individual  expression,  and  promote  divergent  thinking.  The  IGP  is  a 
three  step  process,  which  differs  from  other  brainstorming  methods  by  structuring  an  individual  session  both  before  and. 
after  the  group  brainstorming  activity.  The  individual  sessions  preceding  the  group  session  should  allow  for  cognitive 
rehearsal  and  orientation  to  alleviate  the  effect  of  "production  blocking".  The  succeeding  group  activity  will  allow  for 
social  interactions  and  an  exchange  of  ideas  which  may  serve  as  a "cognitive"  stimulus  for  further  processing  during  the 
final  individual  session.  This  final  individual  session  should  provide  an  opportunity  for  personal  reflection  and 
integration  free  from  perceived  "group  effects",  while  also  accommodating  to  the  proven  productivity  of  idea  generation  as 
an  individual  activity.  Furthermore,  the  emphasis  placed  on  individual  sessions  should  be  more  comfortable  and 
productive  for  "High  Apprehensive"  when  working  in  groups. 
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Purpose  and  Hypothesis 

This  research  synthesizes  many  findings  that  challenge  the  premise  that  Osborn's  traditional  " group" 
brainstorming  treatment  is  the  optimal  method  for  the  generation  of  creative  ideas.  The  purpose  of  this  study  is  to 
investigate  if  an  alternative  form  of  group  brainstorming,  the  IGP,  will  facilitate  a higher  total  production  of  ideas  in 
learning  groups.  One  factor  that  will  be  investigated  is  the  personal  communication  apprehension  level  of  the 
participants,  which  will  be  used  to  group  the  subjects  as  Low  or  High  apprehensives.  These  homogenous  groupings,  in 
accordance  to  communication  apprehension,  will  be  examined  for  the  effect  that  is  caused  by  two  brainstorming 
treatments:  Traditional  group  brainstorming  (Osborn)  and  the  IGP  brainstorming.  Because  the  IGP  places  a greater 
emphasis  on  individual  brainstroming  rather  than  group  brainstorming  sessions,  the  IGP  should  evidence  the  effect  of 
increasing  the  productivity  of  "High  Apprehensive"  individuals.  The  IGP  treatment  should  not  evidence  any  significant 
effect  on  "Low  Apprehensives",  because  there  is  a period  of  group  interaction,  and  overall  the  nominal  (individualized) 
condition  has  proven  to  be  more  productive. 

HI:  Learning  groups  using  the  IGP  brainstorming  method  will  produce  a higher  total  amount  of  ideas  than  groups 
participating  in  the  traditional  brainstorming  method 

H2:  "High  Apprehensive"  individuals  will  produce  an  increased  number  of  ideas  using  the  IGP  brainstorming  method 
than  will  "High  Apprehensive"  individuals  participating  in  the  traditional  brainstorming  method. 

Literature  Review 

Group  Effect 

Beginning  with  Alex  Osborns  original  version  of  his  popular  book.  Applied  Imagination:  Principles  and 
Procedures  of  Creative  Thinking  (1953),  his  process  of  group  brainstorming  continues  to  be  the  technique  which  is  most 
frequently  applied  for  the  generation  of  creative  ideas.  Because  it  is  believed  to  generate  a diversity  of  solutions  to 
problem-solving  situations,  Osborn's  group  brainstorming  is  also  commonly  implemented  to  initiate  many  group 
decision  making  processes.  The  popularization  of  this  method  is  based  on  early  laboratory  studies  at  the  University  of 
Buffalo's  Creative  Problem  Solving  Institute  which  provided  strong  support  for  Osborn's  claims  that : "the  average 
person  can  think  up  to  about  twice  as  many  ideas  when  working  in  a group  than  when  working  alone". 

With  the  continued  interest  in  the  processes  of  creativity,  innovative  thinking,  or  the  formulation  of  challenging 
ideas  and  solutions,  Osborn's  brainstorming  technique  has  important  implications  for  learning  groups.  Does  this  method 
of  group  brainstorming  offer  the  most  effective  method  to  promote  the  expression  of  a diversity  of  ideas  and  individual 
perspectives?  Does  this  process  nurture  the  potential  for  divergent  as  opposed  to  convergent  thinking? 

Osborn  created  his  brainstorming  procedure  as  a method  to  overcome  what  he  believed  was  the  primary  group 
inhibition,  public  disclosure  and  the  fear  of  potential  group  censorship.  He  suspected  the  premature  evaluation  of  ideas 
and  critical  judgments  caused  a fear  of  social  acceptance.  These  apprehensions  blocked  the  firee-flow  of  ideas,  which 
Osborn  felt  is  the  fundamental  process  for  unique  and  creative  ideation.  Osborn  believed  that  a key  component  of  the 
creative  process  is  the  unrestrained  generation  of  a quantity  or  "fluency"  of  ideas.  Therefore,  a major  purpose  of  Osborn's 
brainstorming  technique  was  to  overcome  "group  inhibitions"  by  creating  a risk-free  environment  which  valued  the  fluid 
expression  of  ideas  with  no  fear  of  censorship.  "The  average  group  can  produce  nearly  ten  times  as  many  ideas,  in  the 
same  length  of  time,  when  ideation  is  unhampered  as  when  judgment  is  allowed  concurrently  to  interfere".  Thus,  group 
inhibitions  which  are  caused  by  the  early  evaluation  of  ideas  are  the  fundamental  reason  for  his  four  basic  rules  to 
successful  brainstorming: 

1 . Judicial  judgment  is  ruled  out.  Criticism  of  ideas  must  be  withheld  until  later. 

2.  "Free-Wheeling"  is  welcomed.  The  wilder  the  ideas  the  better;  it  is  easier  to  tame  down  than  to  think  up. 

3.  Quantity  is  wanted.  The  greater  the  number  of  ideas,  the  more  likelihood  of  winners. 

4.  Combination  and  improvement  are  sought.  In  addition  to  contributing  ideas  of  their  own,  participants  should 

suggest  how  ideas  of  others  can  be  turned  into  better  ideas;  or  how  two  or  more  ideas  can  be  joined  into  still 
another  idea  ("Hitch-hiking"). 

Osborn's  claims  were  first  challenged  in  a landmark  study  by  Taylor,  Berry  and  Block  (1957)  who  conducted  an 
experiment  for  the  Department  of  Naval  Research  in  conjunction  with  the  Department  of  Psychology  at  Yale  University. 
In  this  study,  the  total  amount  of  unreplicated  ideas  produced  by  "real"  interacting  groups  comprised  of  four  members,  is 
compared  to  the  total  productivity,  or  "ideation  fluency",  of  four  individuals  who  are  designated  as  a "nominal"  group. 
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Nominal  groups  are  composed  of  individuals  who  work  alone  to  generate  ideas  about  the  same  problem,  and  are  then 
randomly  assigned  to  a "nominal"  group.  The  nominal  group's  performance  is  measured  as  the  sum  total  of  unique  ideas 
generated  by  each  individual  group  member  and  is  "scored  as  though  the  members  had  worked  together  . Taylor  et  al 
concluded  that: 


The  performance  of  the  twelve  real  groups  is  markedly  inferior  to  that  of  the  twelve  nominal  groups  in 
terms  of  ideas  produced  and  in  terms  of  unique  ideas  produced.. ..To  the  extent  that  the  results  of  the 
present  experiment  can  be  generalized,  it  must  be  concluded  that  group  participation  when  using 
brainstorming  inhibits  creative  thinking. 

The  authors  postulated  that  these  results  were  the  product  of  the  group  inhibitions  that  Osborn  sought  to 
overcome,  and  that  despite  his  "rules"  to  prevent  the  preliminary  criticisms  of  ideas: 

Nevertheless  it  appears  probable  that  the  individual  working  in  a group  feels  less  free  of  possible 
criticism  by  others  even  when  such  criticism  is  not  expressed  at  the  time  than  does  the  individual 
working  alone.  To  the  extent  that  this  is  true,  group  participation  will  be  inhibiting.  Group 
participation  may  reduce  the  number  of  effective  ideas  for  a second  reason.  A given  number  of 
individuals  working  in  a group  appear  more  likely  to  pursue  the  same  train  of  thought  - to  have  the 
same  set  or  the  same  approach  to  the  problem  - than  do  the  same  number  of  individuals  working  alone. 
(Taylor,  Berry  and  Block,  1957) 

Dunette,  Campbell  and  Jastaad  (1963)  replicated  the  experiment  by  Taylor  et  al  and  included  in  their  treatment 
that  all  subjects  would  participate  in  both  individual  and  group  brainstorming  sessions.  Their  results  supported  the 
claims  of  the  Taylor  study,  finding  that : "Of  the  24  groups,  only  one  failed  to  produce  more  ideas  under  the  individual 
condition  than  under  the  group  condition"  and  "Only  5 of  the  48  research  subjects  failed  to  produce  more  ideas  when 
working  individually  than  when  participating  in  a group."  They  further  stated  that  four  persons  working  individually 
would  produce  30%  more  ideas  than  individuals  working  together  in  an  interacting  group.  Dunette  et.  al.  concurred  with 
the  Taylor  study’s  explanation  that  groups  tend  to  "fall  in  a rut"  and  members  tend  to  pursue  a similar  train  of  thought 
which  results  in  a convergence  of  ideas.  This  conformity  to  a group  norm,  or  group-think  , negates  the  potential 
productivity  that  stems  from  the  diversity  of  individual  perspectives.  Dunette,  Campbell  and  Jastaad  also  noted  that 
group  inhibitions  cause  certain  individuals,  who  were  highly  productive  when  working  alone,  to  have  a drastic  decline  in 
productivity  when  working  in  a group  condition.  This  led  them  to  the  conclusion  that  despite  Osborn’s  rules  to  prevent 
idea  discrimination  through  criticism: 

It  appears,  however,  that  group  participation  still  contains  certain  inhibitory  influences  which  are  not 
easily  dissipated.  The  "best  bet"  for  creative  thinking  in  attacking  problems  seems,  therefore,  to  be  the 
pooled  individual  efforts  of  many  people  with  perhaps  an  initial  group  session  to  serve  simply  as  a 
warm  up  to  their  (individual)  efforts. 

(Dunette,  Campbell  and  Jastaad,  1963) 

Numerous  other  studies  have  continuously  replicated  the  results  of  the  studies  by  Taylor  et.  al.  and  Dunette  et. 
al.,  evidencing  that  individuals  consistently  outperform  groups  at  generating  ideas:  Vroom,  V.,  Grant,  L.,  and  Cotton,  T. 
(1969);  Bouchard  (1969);  Bouchard,  T.  and  Hare,  M (1970);  Street,  W.  (1974);  Jablin,  F„  Seibold,  D„  and  Sorenson,  R. 
(1977)’,  and  Andre.  M„  Schumer,  H„  and  Whitaker,  P.  (1979).  This  collection  of  research  results  ■ upports  the 
proposition  that  there  docs  exist  some  "group  effect".  Actually,  the  very  premise  of  Osborn’s  original  brainstorming 
rules  are  based  on  his  supposition  that  the  group  effect  of  public  evaluation,  through  intra-group  criticism,  is  a prevalent 
group  condition  and  indeed  inhibiting.  However,  these  other  studies  indicate  that  despite  Osborn's  attempts  to  create  a 
risk  free  and  value  free  environment,  some  group  inhibitory  effect  remained. 

Bouchard  (1969)  continued  this  trail  of  investigations  to  identify  this  "group  effect'  by  examining  the 
communications  apprehension  levels  of  individuals  participating  in  group  brainstorming.  One  should  not  forget  the 
very  strong  possibility  that  personality  type  may  interact  with  both  problem-type  and  group-problem-solving  procedure." 
He  began  one  of  his  series  of  experiments  by  administering  the  California  Personality  Inventory  (CPI)  in  order  to 
examine  the  relationship  between  scores  on  this  measure  and  performance  in  group  ideation  situations. 


Bouchard  concluded  that  one  scale,  Sociability,  and  one  factor.  Interpersonal  Effectiveness,  consistently  related  to 
group  performance.  He  found  that  this  factor  was  "fittingly  enough  developed  to  predict  social  participation"  and  "Thus 
there  is  no  question  that  Interpersonal  Effectiveness  is  a powerful  predictor  of  problem-solving  effectiveness  in  small 
groups.”  Through  a factor  analysis,  Bouchard  found  a significant  relationship  of  p=.89  between  individuals  who  are 
highly  productive  at  idea  generation  in  a group  situation,  and  who  also  have  high  scores  on  the  CPI's  Sociability  scale. 
This  agreement  is  in  accordance  with  the  CPI  developer,  Harrison  Gough's  characterization  of  high  scorers  on  the 
Sociability  scale  as  being:  "outgoing,  enterprising,  and  ingenious,  as  being  competitive  and  forward,  and  as  original  and 
fluent  in  thought."  Subjects  with  high  scores  on  the  CPI  could  be  described  as  "Low  Apprehensive"  in  regards  to  group 
interactions  and  are  very  productive  group  brainstormers.  Bouchard  provides  this  description  of  those  individuals: 

High  scoring  subjects  in  the  brainstorming  groups  have  well  developed  social  skills,  are  outgoing, 
enterprising,  original,  verbally  fluent,  fluent  in  thought,  somewhat  aggressive,  dominant,  and 
controlling,  yet  concerned  with  feelings  of  others.  They  possess  self-assurance,  and  are  spontaneous, 
expressive,  and  enthusiastic.  (Bouchard,  1969) 

Bouchard  believed  that  individual  personality  variables  must  be  taken  into  account  when  seeking  predictions  or 
establishing  groupings  of  individuals.  Consideration  must  be  given  to  the  fact  that  the  social  interactions  of  a group 
process  will  effect  individuals  differently,  and  this  is  dependent  on  the  personal  characteristics  of  Sociability  and 
communication  apprehension.  He  concluded  that  the  group  interactions  between  individuals  of  discrete  communication 
apprehension  levels,  as  indicated  by  differential  CPI  scores,  was  a significant  "group  effect"  that  could  explain  the 
reduction  of  a group's  total  ideation  productivity  (fluency). 

Jablin,  F.,  Seibold,  D.,  and  Sorenson,  R.  (1977)  continued  Bouchard's  investigation  into  personal 
communications  apprehension  as  being  an  appropriate  predictor  of  individual  proficiency/potential  in  subsequent  group 
brainstorming  ideation.  They  administered  McCroskey's  Personal  Report  of  Communications  Apprehension  (PRCA)  as 
a pre-test  measure  of  each  individual  subject's  verbal  communication  level.  This  study  confirmed  an  earlier  study  by 
Japlin  and  Sussman  (1976)  and  indicated  that  the  PRCA  had  a high  correlation  with  the  CPI  (r=.68).  It  also  validated 
Bouchard's  previous  results  and  his  inference  that  groupings  according  to  levels  of  communication  apprehension  would 
result  in  significantly  different  scores,  based  on  the  total  number  of  ideas  produced  by  the  group. 

Japlin  et.al.,  created  homogenous  groups  of  Low  Apprehensives  (LA)  and  High  Apprehensives  (HA)  in 
accordance  to  scores  on  the  PRCA.  The  other  factor  was  the  condition  of  grouping  subjects  into  history  (experience 
working  together  as  a group),  ad  hoc  (newly  formed  homogenous  groups),  or  nominal  groups.  Their  results  indicate  that 
in  every  condition  the  homogenous  groups  of  Low  Apprehensives  produce  more  ideas  than  the  homogenous  groups 
composed  of  High  Apprehensives.  The  fact  that  this  is  a group  effect  is  supported  by  results  that  show  no  significant 
difference  between  LAs  and  HAs  when  measuring  idea  production  in  a nominal  group  condition.  In  this  condition, 
individuals  work  alone  and  thus  there  is  no  effect  caused  by  group  inhibitions,  therefore  High  Apprehensives  should 
perform  better  in  nominal  groups  than  in  interacting  groups.  In  fact,  the  results  of  this  experiment  do  show  that  HAs 
performed  best  when  working  alone,  in  the  "nominal"  condition.  LAs  (high  in  sociability)  appear  to  do  best  in  an 
interactive  condition,  and  they  evidenced  a slight  decline  in  idea  generation  while  in  this  same,  "nominal",  or 
individualized  condition. 

This  study  also  provided  an  important  profile  of  High  Apprehensive  individuals,  which  served  as  an  adjunct  to 
Bouchard's  profile  of  Low  Apprehensives.  High  Apprehensives  will  perform  best  in  situations  where  there  is  little 
demand  placed  on  oral  communication  and  participation.  High  Apprehensives  can  be  described  as  tending  to  participate 
less  in  group  discussions  and  having  little  trust  in  the  communicative  behavior  of  others.  They  normally  shun 
competitive  transactions  and  are  generally  not  effective  in  social  relationships.  In  social  situations  they  are  apologetic 
about  their  ideas  and  prefer  writing  to  oral  communication.  In  general:  "the  individual  who  is  high  in  communication 
apprehension  is  one  for  whom  the  negative  consequences  attached  to  participating  in  an  oral  interchange  outweigh  any 
perceived  gains". 

Japlin  et.al.  provided  additional  information  to  support  the  premise  that  participation  in  a group  brainstorming 
session  will  effect  individuals  differently  in  accordance  to  their  level  of  Sociability  or  communication  apprehension.  It 
also  supports  the  theory  of  a "group  e.Tect",  which  is  manifested  by  reducing  the  productivity  of  HA  individuals  who  arc 
effected  by  the  condition  of  working  in  a gtoup.  Japlin  et.  al.  consistently  found  that  High  Apprehensives  performed 
better  in  a nominal  (individual)  brainstorming  condition  and  Low  Apprehensives  performed  best  in  an  intei  acting 
condition.  These  results  suggest  that  how  well  an  individual  performs  in  a group  brainstorming  session  appears 
dependent  upon  one’s  predisposition  toward  group  interactions.  The  "group  effect"  will  have  a differential  effect, 
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depending  on  each  individual’s  trait  of  communications  apprehension,  which  will  effect  their  capability  vo  perform  in  a 
group  condition.  This  lead  Japlin  et.  al.  to  the  conclusion  that  one  factor  that  effects  group  brainstorming  performance  is 
a “personality  characteristic": 

Effective  group  brainstormers  tend  to  be  high  in  Sociability,  low  on  Communications  Apprehension, 
and  generily  effective  in  interpersonal  interactions.  To  the  extent  that  these  traits  are  missing  in  group 
brainstorming,  members’  performance  may  be  inferior  to  working  alone.  (Japlin  and  Seibold,  1978) 

Brainstorming  Treatment 

There  have  been  numerous  other  studies  which  have  investigated  the  possible  sources  of  a “group  effect”.  Street 
( 1 974)  concurred  with  Taylor  et.  al.  that  despite  Osborn's  brainstorming  rules  to  prevent  the  critical  evaluation  of  ideas 
until  later,  individuals  appear  to  exhibit  a "fear  of  social  chastisement"  by  the  other  group  members.  “In  interacting 
groups,  members  censor  their  contributions  because  they  fear  possible  social  disapproval  by  the  other  group  members." 
Vroom,  Grant,  and  Cotton  (1969)  further  confirmed  the  assumptions  by  Taylor  et.  al.  and  Dunette  et.  al.,  that  the  range 
of  different  ideas  in  a brainstorming  group  appears  to  be  limited  by  the  "motivational  pressure  to  conform".  Nominal 
groups  consistently  produce  more  unique,  diverse  and  creative  ideas  than  do  interacting  groups.  Vroom  and  colleagues 
interpreted  these  findings  as  an  indication  that:  " interaction  tends  to  result  in  members  developing  a common  set  in  their 
approach  to  the  problem*  which  is  suggestive  of  'cognitive  inertia'." 

Lamm  and  Trommsdorf  (1973)  continued  investigations  into  the  “group  effect"  and  identified  "production 
blocking"  as  another  important  aspect  of  this  overall  effect.  Their  findings  lead  them  to  the  conclusion  that:  "the  most 
important  source  of  the  inferiority  of  groups.. ..is  the  operation  of  the  implicit  rule  that  only  one  member  of  the  group 
speaks  at  a time".  This  causes  a reduction  in  overall  group  productivity  because  "the  production  time  theoretically 
available  to  each  group  member  of  a four-man  group  would  be  one-fourth  of  that  available  under  the  individual 
conditions." 

Recent  investigations  by  Diehl  and  Stroebe  (1991)  have  advanced  the  investigation  of  "production  blocking". 
They  conducted  a series  of  experiments  which  compare  the  difference  between  three  "group  effects"  identified  by  previous 
research  as  being:  free  riding  (social  loafing),  evaluation  apprehension  (fear  of  chastisement),  and  production  blocking. 
The  analysis  of  these  experiments  lead  them  to  the  conclusion  that  the  effect  of  "production  blocking"  is  the  most 
significant  influence  on  group  performance. 

Production  blocking  is  caused  by  the  dynamics  of  group  interactions  which  causes  interference  with  an 
individual's  cognitive  processes.  The  fact  that  another  individual  in  the  group  is  expressing  his/her  ideas  often  means  that 
other  group  members  have  to  "wait  their  turn"  before  they  can  express  their  thoughts.  Diehl  and  Stroebe  felt  that 
"production  blocking"  effects  a group  member's  productivity,  because  this  "delay"  in  the  process  causes  individuals  to 
devote  their  short-term  memory  to  "cognitive  rehearsal"  of  their  own  ideas.  Therefore,  in  order  to  prevent  themselves 
from  forgetting  their  own  ideas  while  "waiting  their  turn",  group  members  invest  most  of  their  cognitive  energies  into 
remembering  those  ideas  which  they  previously  created.  This  prevents  them  from  developing  new  ideas  because:  "After 
all,  storage  space  in  short  term  memory  is  fairly  limited,  and  individuals  will  only  be  able  to  store  a small  number  of 
ideas  at  a given  time."  (Diehl  and  Stroebe,  1987) 

Indeed,  Diehl  and  Stroebe  (1991)  found  the  production  blocking  effect  to  account  for  96%  of  the  total  variance  in 
idea  generation.  They  found  that:  "When  subjects  were  able  to  report  their  ideas  as  soon  as  they  occurred,  they  produced 
approximately  twice  as  many  ideas  as  when  they  had  to  wait  for  other  speakers  to  finish  talking." 

The  authors  a>so  felt  that  the  exposure  to  other  group  member's  ideas  caused  a "distraction-conflict"  which 
prevented  individuals  from  devoting  cognitive  energy  toward  the  development  of  new,  alternative  ideas.  Instead,  mental 
cognition  becomes  devoted  to  understanding  and  interpreting  the  ideas  of  others  in  the  group.  If  group  members  are 
listening  to  others,  and  simultaneously  trying  to  take  advantage  of  the  waiting  time  as  an  opportunity  to  create  new 
ideas,  the  resulting  distraction  causes  "cognitive  overload".  Diehl  and  Stroebe  found  that  providing  group  members  with 
a notepad  to  externally  store  their  ideas  is  a helpful  memory  adjunct.  However,  because  the  very  nature  of  group 
dynamics  (listening  to  other  ideas/generating  new  ideas)  still  cause  a "distraction-conflict",  they  recommend  that:  "it 
might  be  more  helpful  to  ask  subjects  to  develop  their  (written)  ideas  in  individual  sessions"  before  entering  the  group 
condition. 

Dunette,  Campbell  and  Jastaad  (1963)  found  that  group  participation  appeared  to  provide:  "the  important 
function  of  a 'warm-up'  for  subsequent  brainstorming  activity".  They  believed  there  is  an  important  motivational  aspect 
to  group  participation  which  is  caused  by  social  interactions  and  social  processing.  The  group  dynamics  of  exchanging 
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perspectives  causes  cognitive  arousal  and  increased  stimulation,  while  the  very  act  of  group  participation  causes  increased 
intrinsic  motivation.  Their  research  indicates  that  individuals  produced  more  information  when  the  "group  stimulation" 
is  followed  by  a focused  individual  brainstorming  session. 

A larger  number  of  ideas  or  solutions  was  produced  when  subjects  experienced  the  individual 
brainstorming  after  having  experienced  the  group  session,  than  when  they  "went  in  cold"  to  the 
individual  session. 

(Dunette,  Campbell  and  Jastaad,  1963) 

Andre,  Schumer  and  Whitaker  (1979)  also  found  that  subjects  in  their  experiment  produced  more  ideas  when  the 
group  brainstorming  preceded  an  individual  session.  Their  results  suggest  that:  "while  group  experience  inhibits 
creativity  for  discussed  items  it  may  serve  as  a spur  to  individual  creativity  for  subsequent  items."  They  believe  that  this 
is  caused  by  a "lowering  of  the  psychological  barriers"  in  the  individual  by  exposing  them  to  the  "wild  ideas  of  other 
group  members".  This,  in  effect,  lowers  the  threshold  of  expected  social  chastisement  and  provides  a safer  atmosphere  for 
individual  creativity.  A group  norm  becomes  established  that  allows  individuals  to  be  more  receptive  to  novel  or  unusual 
ideas,  and  this  stimulates  the  individual  to  more  freely  express  themself. 

IGP  Brainstorming  Technique 

This  previous  research  suggests  that  perhaps  Osborn's  traditional  group  brainstorming  process  itself  needs  to  be 
altered,  in  order  to  take  advantage  of  both  the  individual  process  and  group  process  of  idea  production.  Most  of  the 
research  indicates  that  the  most  productive  generation  of  ideas  occurs  when  individuals  brainstorm  by  themselves  and  then 
combine  their  ideas  in  a "nominal  group".  I am  proposing  an  alternative  method  of  brainstorming  called  the  IGP 
(Individualized-Grouped-Personalized),  which  will  allow  the  participants  to  experience  the  benefits  of  both  individual  and 
grouped  sessions.  It  is  similar  in  every  way  to  Osborn's  original  brainstorming  rules,  except  that  it  allows  for  an 
individual  session  both  before  and  after  the  group  brainstorming  activity.  The  overall  design  of  the  IGP  begins  with  an 
individual  session,  followed  by  a group  session,  and  ending  with  a follow  up  individual  session.  This  allows  for  more 
emphasis  on  the  individual  sessions  which  have  proven  to  be  more  productive,  but  also  uses  the  group  activity  to 
stimulate  further  individual  processing. 

It  is  hoped  that  the  individual  sessions  preceding  the  group  session  will  allow  for  cognitive  rehearsal  and 
orientation  that  may  alleviate  the  effect  of  production  blocking.  This  will  also  allow  the  group  brainstorming  session  to 
incorporate  the  Nominal  Group  Technique  developed  by  Van  de  Ven  and  Delbecq  (1975)  which  suggests  that  allowing 
individuals  to  write  down  their  ideas  before  a group  session  facilitates:  "self-disclosure  of  ideas,  even  by  less  secure 
members  who  may  hesitate  to  bring  some  problem  dimension  before  the  group  in  the  conventional  interacting  situation." 

The  IGP  should  also  benefit  from  the  "warm-up"  of  having  a group  session  before  the  final  individual  session. 
This  will  serve  to  stimulate  individual  productivity  as  a function  of  the  dynamics  of  group  interactions  and  the  cognitive 
arousal  caused  by  exposure  to  new  and  challenging  ideas.  The  final  individual  session  should  benefit  from  the  arousal  and 
increased  intrinsic  motivation  caused  by  the  previous  group  participation.  However,  it  will  also  allow  individuals  to  be 
free  from  the  distraction-conflict  that  is  often  caused  by  group  interactions  and  which  results  in  cognitive  overload. 

It  should  also  be  noted  that  the  IGP  is  constructed  to  allow  for  individual  differences;  the  personality 
characteristics  that  are  indicated  by  personal  communication  apprehension  scores.  The  group  session  should  be  conducive 
to  "Low  Apprehensives"  who  generally  produce  mo:e  ideas  when  in  a condition  of  interpersonal  interactions.  While  the 
overall  emphasis  on  individual  sessions  will  be  less  intimidating  and  more  productive  for  "High  Apprehensive" 
individuals,  who  frequently  find  the  group  process  less  comfortable  and  productive.  Overall,  the  IGP  attempts  to 
incorporate  the  way  individuals  learn  by  social  interaction,  as  well  as  the  way  learning  occurs  through  the  process  of 
personal  cognition. 

The  purpose  of  this  experiment  is  to  determine  if  an  alternative  method  of  brainstorming,  the  IGP,  will  enable 
learning  groups  to  increase  the  total  amount  of  ideas  they  produce.  By  integrating  the  proven  benefits  of  an 
individualized  (nominal)  condition  with  the  suspected  cognitive  benefits  of  group  interactions,  the  IGP  should  provide  a 
more  productive  brainstorming  condition  than  does  the  traditional  group  brainstorming  method.  The  IGP  is  an  attempt 
to  synthesize  the  process  of  social  learning  with  the  process  of  individual  creativity.  This  not  only  takes  into  account 
learner  preferences,  as  evidenced  by  their  communication  apprehension  traits,  but  also  provides  individual  sessions  to 
alleviate  the  apparent  variety  of  "effects"  associated  with  group  participation. 
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The  PRCA  test  has  seldom  been  used  with  younger  learners,  but  it  should  provide  valuable  information  in 
regards  to  individual  learning  styles  that  deserve  consideration.  I am  blocking  for  levels  of  communication  apprehension 
to  control  for  their  documented  differences  in  regards  to  group  brainstorming  ideation.  I also  want  to  examine  if  the  two 
different  types  of  brainstorming  treatment  evidence  any  differential  effect  in  regards  to  individual  levels  of 
communication  apprehension.  It  is  suspected  that  the  IGP  treatment  should  enhance  the  productivity  of  High 
Apprehensives  because  of  the  IGP’s  emphasis  on  individual  sessions.  However,  the  IGP  should  also  improve  overall 
group  productivity  by  synthesizing  the  creative  experiences  of  both  group  processing  and  the  individual  process  of 
reflection  or  personal  cognition. 


Methods 

Subjects 

Participants  were  108  seventh  graders  (49  female/59  male)  from  an  urban  Midwestern  public  middle  school.  All 
were  enrolled  in  mixed  ability  English  composition  classes,  where  group  brainstorming  is  introduced  as  a technique  for 
the  creative  generation  of  ideas. 

Materials 

A workshop  was  created  to  provide  a training  session  on  the  activity  of  brainstorming,  as  well  as  present 
practice  exercises  using  both  the  brainstorming  treatments.  The  “Traditional  Rules  of  Brainstorming"  were  based  on 
Osborn's  four  main  concepts: 

1 . Do  not  criticize  any  ideas 

2.  All  types  of  ideas  are  encouraged  - the  wilder  the  better 

3.  The  goal  is  to  create  as  many  ideas  as  possible 

4.  "Hitch-hiking":  using  a combination  of  someone  else's  ideas  to  create  new  ones  is  encouraged.  The 

facilitator  will  write  down  all  the  group  ideas. 

The  "Rules  of  the  IGP  Brainstorming"  were  based  on  the  above  four  concepts  and  included  the  following  procedures: 

1.  You  will  first  brainstorm  individually  for  5 minutes.  During  this  session  there  will  be  no  talking 
and  you  need  to  write  down  all  the  ideas  that  you  think  of.  Keep  your  "idea  sheet". 

2.  At  the  end  of  5 minutes  the  facilitator  will  come  around  and  bring  you  together  in  your  group.  At 
this  time  draw  a line  across  the  page  below  your  last  individual  idea. 

3.  You  will  be  in  the  group  for  10  minutes  and  at  this  time  you  are  encouraged  to  talk  with  other 
group  members,  share  your  ideas,  and  create  new  ideas  with  the  group.  The  facilitator  will  write  down 
all  the  group  ideas. 

4.  At  the  end  of  the  group  session  you  will  return  to  your  seats  and  individually  brainstorm.  Use  any 
group  ideas  to  help  create  new  ones  (hitch-hike).  Write  down  all  the  ideas  that  come  to  yon,  including 
any  "repeats"  on  your  "idea  sheet".  At  the  end  of  5 minutes  the  facilitator  will  collect  your  idea  sheet . 

The  PRCA  pre-test  is  a valid  and  reliable(.93)  measure  of  communication  apprehension  (McCroskey,  1970,1985)  and  an 
indicator  of  individual  productivity  in  brainstorming  groups  (Jablin  et.  al.,  1977).  The  problems  used  in  this  research 
have  also  been  validated  as  appropriate  brainstorming  exercises  (Thornburg,  1991 ; and  Renzulli,  Owens,  and  Callahan, 
1974). 

Design  and  Procedures 

One  week  before  the  study,  ail  students  were  given  a workshop  devoted  to  the  purposes,  instructions  and 
procedures  of  the  two  brainstorming  treatments.  Next  all  participants  were  administered  the  PRCA  to  assess  their 
individual  levels  of  communication  apprehension.  The  PRCA  scores  were  used  to  divide  students  into  high  or  low 
apprehensives  based  on  their  scores  in  relationship  to  the  test  median  (60).  Participants  with  scores  at  or  above  this 
median  were  categorized  as  "High  Communication  Apprehensives".  Members  from  these  two  apprehension  categories 
were  then  randomly  assigned  to  homogenous  groups  of  four,  and  these  groups  were  randomly  assigned  to  either  a 
Traditional  (Osborn's  groups)  or  IGP  brainstorming  treatment  in  a 2x2  factorial  design. 

During  the  individual  brainstorming  sessions  of  the  IGP,  subjects  wrote  down  all  their  ideas;  during  all  group 
sessions  the  subjects  orally  shared  their  ideas  and  those  were  recorded  by  multiple  raters.  The  subjects  were  asked  to 
generate  as  many  ideas  as  possible  in  20  minutes  on  two  exercises  which  have  been  previously  validated  as  adequate  for 
full  ideation  (Bouchard,  1969  and  Diehl  and  Stroebe,  1991).  The  final  group  scores  are  based  on  "fluency"  - the  total 
number  of  unreplicated  ideas  that  each  group  generated  within  the  20  minute  period. 
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Results 

The  results  were  subjected  to  an  analysis  of  variance  for  uneven  cell  size  with  a one-way  ANOVA.  Blocking 
was  done  for  levels  of  communication  apprehension  from  the  results  of  the  PRCA  pre-test  (49  HA  and  59  LA).  Of  the 
49  HA  individuals  the  majority  were  found  to  be  female(30/49).  The  ANOVA  produced  no  significant  differences  for 
Treatment  E(L  22)=2.38,  p<.089;  Communication  Grouping  £(1,  22)=1.50,  p<.236;  Interaction  E(l,  22)=0.855, 
p<.367. 


Discussion 

While  there  were  no  statistically  significant  results  the  study  did  provide  some  interesting  findings  that  will  be 
useful  for  future  studies.  Although  placing  participants  in.groups  of  four  dominates  the  existing  body  of  research,  there 
is  no  evidence  that  this  is  a necessary  condition,  except  that  it  concurs  with  the  precedent  set  by  Taylor  et.  al.  (1957). 
Using  groups  of  four  students  greatly  diminished  the  overall  power  of  this  study  and  subsequent  research  has  indicated 
that  smaller  group  sizes  may  prove  to  be  equally  valid  (Thornburg,  1991;  Renzulli  et.  al.  1974).  This  may  have 
particularly  influenced  the  significance  of  the  IGP  treatment  effect  because  the  means  for  the  two  brainstorming  exercises 
all  appear  to  indicate  a difference  regardless  of  the  communication  level:  Question  1 - IGP/81.6  vs.  Traditional/62.6; 
Question  2 - IGP/84.4  vs.  Traditional/  69.0.  Surprisingly,  no  difference  was  found  between  the  two  communication 
levels,  however  the  LAs  did  consistently  outperform  the  HAs  and  the  lack  of  statistical  power  may  have  also  influenced 
this  outcome.  There  was  an  indication  of  a slight  trend,  where  the  difference  between  the  means  of  HAs  and  LAs 
diminished  from  the  first  and  second  exercise  (Q1  - HA/61.7  vs.  LA/84.6  and  Q2  - HA/72.3  vs.  LA/81.9).  This  might 
suggest  that  HA’s  become  more  socially  comfortable  over  time  which  positively  effects  their  performance. 

Another  factor  which  may  have  diminished  the  effect  of  the  IGP  was  the  length  of  time  given  for  the  treatment. 

In  its  original  design,  the  IGP  would  allow  more  time  between  the  three  sessions.  The  first  individual  session  would 
occur  a few  days  before  the  group  session,  providing  an  opportunity  for  the  individual  generation  of  ideas  as  well  as  an 
advanced  organizer.  After  the  group  brainstorming  session,  participants  would  be  given  a copy  of  all  the  ideas  generated 
up  to  that  point  and  then  allowed  a few  days  for  further  ideation.  It  is  believed  that  the  twenty  minute  sessions  provided 
in  this  research  study  is  an  "artificial"  constraint  on  the  cognitive  processes  of  orientation  and  reflection.  However,  the 
longer  treatment  would  be  problematic  in  a school  class  setting  and  little  control  on  providing  equal  "time  on  task" 
between  the  two  treatments.  Future  research  might  be  conducted  using  the  "ideal"  IGP  treatment  condition  and  allowing 
for  three  separate  group  sessions,  even  though  this  might  mean  sacrificing  experimental  control  of  the  time  variable. 

The  results  of  the  PRCA  pre-test  suggests  that  gender  would  be  an  interesting  factor  to  investigate  both  in 
regards  to  communication  apprehension  and  brainstorming  performance.  The  age  and  nature  of  this  population  sample 
may  also  have  adversely  effected  the  research  results.  Previous  brainstorming  studies  have  usually  been  performed  with 
an  older  population,  who  are  more  familiar  with  brainstorming  and  group  work.  Future  studies  may  also  benefit  from 
more  contextually  relevant  brainstorming  exercises  concerning  personally  meaningful  problems  and  a vested  interest  in 
the  successful  discovery  of  potential  and  alternative  solutions. 

Although  it  was  a purposeful  intention  of  this  study  to  examine  brainstorming  in  a "natural"  social  setting,  and 
thus  avoid  the  confounding  effects  of  an  electronically  mediated  experience,  another  area  for  future  investigation  is  the 
effect  of  various  computer  based  technologies  (Networks  and  GDSS).  Many  studies  have  indicated  the  effectiveness  of  a 
GDSS-type  interface  and  perhaps  merging  these  technologies  with  a strategy  such  as  the  IGP  will  further  enhance  idea 
production.  Some  advantages  of  a computer  interface  would  be  their  capability  to  overcome  various  forms  of  group 
inhibitions:  an  anonymous  individual  condition  to  negate  the  effects  of  evaluation  apprehension;  an  efficient 
management  of  the  group  process  through  the  synchronous  expression  of  ideas  (production  blocking);  an  efficient  record 
keeping  mechanism  of  individual  contributions.  However,  since  many  schools,  organization  and  groups  do  not  have 
access  to  this  type  of  equipment  it  still  remains  pertinent  to  observe  the  effects  of  the  IGP  without  an  electronic  system. 

As  an  area  of  research,  group  brainstorming  is  an  interesting  example  of  a populrriy  employed  group  activity 
which  is  widely  believed  to  increase  productivity,  yet  these  actual  performance  gains  are  unsubstantiated  by  existing 
research.  Group  brainstorming  is  a classic  paradox  concerning  the  appropriate  use  of  group  vs.  individual  efforts,  and 
thus  it  reveals  many  questions  as  to  "what"  factors  have  a negative  effect  on  group  performance.  Is  it  the  inherent  "nature 
of  the  task"  itself,  and  what  are  the  specifics  of  the  task  demands  that  perhaps  makes  a particular  activity  more  appropriate 
and  productive  for  individual  endeavors  as  opposed  to  group  work?  Is  it  the  various  effects  of  group  composition  and 
individual  differences  on  group  performance?  Are  the  negative  influences  of  "group  effects"  an  inevitable  consequence  of 
group  work  and  what  group  management  strategies  can  be  developed  to  overcome  these  problems? 
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We  do  philosophy  (theorize)  when,  for  whatever  reason,  we  are  aroused  to  wonder  about  how  events  and  experiences 
are  interpreted  and  should  be  interpreted.  We  philosophize  when  we  can  no  longer  tolerate  the  splits  and  fragmentations  in  our 
pictures  of  the  world,  when  we  desire  some  kind  of  wholeness  and  integration,  some  coherence  which  is  our  own  (Greene, 
1974,  pp.  10-1 1).  This  is  private  work.  This  happens  on  a personal  level  as  we  work  through  the  dailiness  of  everyday  life. 

It  becomes  public  when  we  struggle  with  the  realities  of  classroom  life. 

As  educators,  we  are  concerned  with  philosophical  issues  and  questions  in  our  daily  work  within  classrooms. 

As  we  debate  curricular  issues,  as  we  decide  educational  policy,  as  we  work  with  students  and  their  "behavior",  as  we 
"test*1  students’  ’’knowledge”,  etc.,  we  are  concerned  with  philosophy.  However,  the  underlying  ideas  behind  our  practice 
may  go  un-noticed,  may  be  unconscious,  may  be  unquestioned.  The  importance  of  philosophical  inquiry  in  education  is 
exactly  at  this  point:  philosophical  inquiry  can  illuminate,  inform,  call  into  questi  n,  etc.,  the  taken  for  granted  notions 
that  we  have.  Philosophical  inquiry  and  analysis  can  help  conceptual  clarification,  as  well  as  inform  our  praxis,  and  vice 
versa. 

In  this  paper/presentation,  I will  take  the  position  that  philosophy  is  a critical  way  to  studying  schooling. 

Hence,  the  study  of  philosophy/philosophy  of  education  will  provide  a framework  needed  for  inquiry  into  schooling  that 
is  foundational  (theoretical),  diverse,  and  critical.  This  will  suggest  the  need  for  taking  a theoretical  stance  with  regard  to 
our  work.1 

Taking  a theoretical  stance  with  regard  to  our  work  suggests  dealing  with  a broad  analysis  of  education,  i.e., 
concern  with  "the  big  picture”.  The  big  picture  of  education  is  complicated.  At  the  same  time,  educators  are  confronted 
with  a sense  of  urgency  to  solve  educational  problems.  Philosophy  is  too  time  consuming.  As  Eisner  (1991)  states, 
’’Philosophy  is  nagging.  It  conjoles  students  into  asking  questions  about  basic  assumptions,  it  generates  doubts  and 
uncertainties,  and,  it  is  said,  it  keeps  people  from  getting  their  work  done"  (pp.4-5).  Philosophizing  about  the  issues  does 
take  time,  and  it  is  messy.  And  this  does  seem  to  take  us  away  from  our  work. 


Philosophical  inquiry  provides  ways  for  educators  to  "inquire  into  their  work",  to  question  their  theory  and 
practice.  Processes  of  philosophical  inquiry  include  the  following:  (1)  conceptual  analysis;  (2)  situating  educational 
issues  within  philosophical  traditions;  (3)  the  examination  of  epistemological  and  axiological  assumptions,  criticisms, 
and  (4)  critical  thinking  and  analysis  of  existing  literature  and  theory.  These  processes  are  ways  of  doing  philosophy.2 

I believe  philosophy  is  the  foundation  (theory)  of  educational  research.  This  can  be  seen  by  posing  the  basic 
questions  of  philosophy:  What  is  the  nature  of  reality?  What  is  the  nature  of  knowledge?  What  is  of  value?  These 
questions  provide  a conceptual  framework  that  gives  coherence  to  the  study  of  philosophy.  These  questions  also  identify 
the  major  concerns  of  education  and  provide  the  possibility  for  a coherence  in  educational  practice.  By  coherence  I mean 
they  provide  educators  with  a possible  framework  for  posing  questions  from  multiple  perspectives  that  allow  us  to  reflect 
on  our  work.  For  example,  they  allow  us  to  pose  multiple  questions  regarding  the  nature  of  curricula.  They  allow  us  to 
examine  whose  knowledge  we  are  promoting,  and  even  prior  to  that,  what  knowledge  is  of  most  worth.  Questions  of 
value  ask  us  why  we  choose  this  particular  knowledge,  and  leave  all  of  the  rest  out,  etc.  Engaging  in  this  questioning  is 
philosophical  inquiry,  is  doing  philosophy. 

Another  framework  I can  use  in  looking  at  the  relationship  between  philosophy,  research  and  education,  would 
be  an  examination  of  the  differing  approaches  to  the  study  of  philosophy.  Wingo  (1974)  states  that  we  can  approach  the 
study  of  philosophy  in  three  different  ways  (these  ways  may  also  be  looked  at  as  the  main  functions  of  philosophy):  the 
descriptive,  the  normative,  and  the  analytic  (pp.  15-1 6). 

To  engage  in  descriptive  philosophical  inquiry,  a student  would  be  involved  in  the  study  of  the  history  of 
philosophy.  S/he  would  be  studying  "what  is  (and  has  been)  the  field  of  philosophy.  Working  comprehensively,  he  is 
trying  to  picture  the  general  development  of  philosophical  thought"  (p.15).  This  is  more  than  studying  "intellectual 
history”.  As  Wingo  points  out,  it  is  possible  to  study  about  what  philosophers  have  said,  and  at  the  same  time  be  doing 
philosophy  in  that  we  are  "analyzing  and  clarifying  concepts  and  the  language  in  which  ideas  are  expressed"  (p.  1 5).  This 
is  the  area  identified  earlier  as  situating  educational  issues  within  a philosophical  tradition.  For  example,  educational 
issues  looked  at  from  the  viewpoint  of  different  philosophies,  and  what  writers  within  those  philosophical  traditions  said 
about  the  issues,  how  they  would  go  about  making  sense  of  those  issues,  establish  a world-view  (metaphysics/ontology), 
a way  of  knowing  (epistemology),  and  a way  to  make  decisions  regarding  action  (axiology).  Philosophy  of  education 
textbooks  would  be  good  examples  of  the  descriptive  perspective  (cf.  Wingo,  1974;  Gutek,  1988;  Ozmon  and  Craver, 
1995). 
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To  engage  in  normative  philosophical  inquiry  a student  would  be  involved  with  values  (axiology).  Interests 
could  focus  on  ethics  or  aesthetics.  He  will  be  involved  with  advocating  some  ends  or  objectives  (values)  that  he 
believes  to  be  desirable  and  with  explaining  the  reasons  for  their  desirability.  He  may  also  be  involved  in  suggesting 
means  for  advocating  these  values.  His  main  concern  is  not  what  is,  but  what  ought  to  be.  (p.15)  Normative 
philosophical  inquiry  explores  and  critiques  philosophical  positions,  as  well  as  makes  decisions  as  the  'Tightness  and 
wrongness"  of  those  positions  (see  Webb,  et  al,  1992).  The  normative  perspective  requires  an  examination  of 
epistemological  and  axiological  assumptions,  as  well  as  critical  thinking  and  analysis  of  existing  literature  and  theory  as 
part  of  inquiry  (cf.  Kaufmann,  1974;  Dewey,  1904  [1964]). 

To  engage  in  analytic  philosophical  inquiry  is  to  engage  in  the  "analysis  of  language,  concepts,  theories,  and  so 
on"  (Wingo,  1974,  p.  15).  This  is  the  practice  that  analytic  philosophers  consider  "doing  philosophy".  According  to 
Webb,  et  al.  (1992),  analytic  philosophy  has  as  its  goal  to  improve  our  understanding  of  education  by  clarifying  our 
educational  concepts,  beliefs,  arguments,  assumptions.  For  example,  an  analytic  philosophy  of  education  would  attempt 
to  understand  such  questions  as:  What  is  experience?  What  is  understanding?  What  is  readiness?  (pp.  174-5)  The  analytic 
perspective  has  to  do  with  conceptual  analysis  (cf.,  Tom,  1984;  Wilson,  1963). 

This  framework  of  the  "functions"  of  philosophy  suggests  the  foundations  position  mentioned  earlier.  Each  of 
these  three  functions  can  provide  multiple  possibilities  for  educational  research  and,  more  specifically,  philosophical 
inquiry.  The  descriptive,  normative  and  analytic  forms  of  philosophical  inquiry  suggest  indepth  study  of  the  philosophy 
of  education.  Looking  at  different  philosophical  traditions  with  regards  to  metaphysics/ontology,  epistemology  and 
axiology  requires  study  in  philosophy.  Movements  in  education,  e.g.,  reconstructionism,  perrenialism,  essentialism, 
Marxism  and  education,  and  more  recent  movements  rooted  in  critical  theory,  postmodern  analyses,  renewed  emphasis  on 
democratic  schooling  and  forms  of  emancipatory  praxis,  etc,,  represent  major  areas  of  study  for  researchers.  These  areas 
of  study  have  their  own  world-views  and  concerns.  Writers  within  these  positions  offer  differing  conceptual  frameworks, 
differing  questions  posed,  and  hence  challenges  to  status  quo  practice.  And  they  all  engage  the  student  in  philosophical 
inquiry. 

The  study  of  the  philosophy  of  education  from  the  normative,  descriptive  and  analytic  perspectives  offer  critical 
means  of  inquiry  into  educational  realities  for  researchers.  Writing  within  these  frameworks  is  doing  philosophy.  Doing 
philosophy  is  doing  research.  The  descriptive  perspective  works  out  of  systems  of  philosophical  thought,  schools  of 
thought,  offering  foundational  positions  from  which  to  work;  the  normative  perspective  offers  a "process  of  inquiry  into 
ideas  and  basic  beliefs  that  will  enable  us  to  form  reasoned  attitudes  about  the  important  issues  of  our  time"  (Wingo, 

1974,  p.22).  The  analytic  perspective  allows  us  to  inquire  into  the  use  of  language,  the  meaning,  and  clarification  of 
language  used  to  talk  about  education.  This  is  philosophical  inquiry.  This  is  doing  philosophy. 

Those  of  us  involved  in  education  knov:  that  it  is  a very  complex  social  undertaking.  It  has  many  important 
dimensions  that  can  be  examined  from  psychological,  sociological,  political  perspectives,  yet  there  is  one  question  that  is 
uniquely  philosophical:  "the  question  of  determining  the  ends  of  education"  (Wingo,  1974,  p.22).  The  means  and  ends  of 
education  are  inseparably  united.  Wingo  (1974)  quotes  Max  Black: 

All  serious  discussion  of  educational  problems,  no  matter  how  specific,  soon  leads  to  consideration  of 
educational  aims,  and  becomes  a conversation  about  the  good  life,  the  nature  of  man,  the  varieties  of  experience.  But 
these  are  the  perennial  themes  of  philosophical  investigation. 

It  might  be  a hard  thing  to  expect  educators  to  be  philosophical,  but  can  they  be  anything  else?  (p.22) 
Conceptualizations  about  "the  good  life",  the  nature  of  humankind,  etc.,  are  problematic  because  there  are  no  final,  all 
inclusive  positions  on  these  concepts.  Inquiry  into  these  issues  can  take  place  through  the  descriptive,  normative,  and 
analytic  perspectives.  And  each  of  these  perspectives  will  demand  different  questions  be  posed.  This  process  is  doing 
philosophy,  doing  philosophical  inquiry. 

To  understand  how  the  three  perspectives  can  be  used  in  the  study  of  education,  Wingo  (1974)  suggests  that 
there  are  three  assumptions  that  underlie  the  nature  of  philosophical  inquiry  in  education.  These  three  assumptions  are 
critical  to  an  understanding  of  the  importance,  scope,  and  possibility  the  study  of  philosophy  has  for  the  study  of 
education.  As  obvious  as  it  may  seem,  the  first  assumption  is:  "The  primary  subject  matter  of  philosophy  of  education 
is  education  itself  (p.24).  Thus  the  phenomena  of  education,  in  all  its  myriad  forms,  are  the  "subject  matter"  for  study. 
From  a research  point  of  view  this  can  mean  looking  at  curricula,  the  outcomes  of  learning,  testing,  organizational 
matters,  place  of  schools  within  the  social  setting,  the  means-ends  of  education,  etc.,  etc. 

The  second,  and  perhaps  the  most  insightful  and  critical  assumption,  states  that  "Education  always  takes  place 
within  a certain  constellation  of  cultural  conditions  and  therefore  it  cannot  be  studied  as  a set  of  universal  and  independent 
phenomena"  (p.24).  This  assumption  means  that  there  is  no  "one  best  system"  to  model  schools  after  and  no  single 
answer  to  complex  educational  situations.  This  assumption  suggests  that  we  need  to  view  education  relational^,  in 
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context  (cf  Apple,  1979;  Beyer,  1986;  Purpel,  1993),  and  clearly  identifies  the  complex  nature  of  understanding 
education.  At  the  same  time  this  assumption  suggests  the  myriad  possibilities  for  inquiry  into  the  process  of  schooling. 
By  this  I mean  that  the  nature  of  the  inquiry  is  dependent  upon  the  researcher.  It  is  not  standardized,  it  is  not  given  (cf. 
Eisner,  1991). 

The  third  assumption  underlying  the  nature  of  philosophical  inquiry  in  education  states  that  the  "basic  purpose 
of  philosophy  of  education  applies  to  the  ends  and  means  of  education  and  their  interrelationships"  (Wingo,  1974,  p.24). 
The  assumption  suggests  the  complexity  of  educational  experience  and  the  many  variables/  factors  that  influence  the 
process.  These  assumptions  clearly  call  for  the  descriptive,  normative,  and  analytic  perspectives  of  viewing  educational 
realities,  but  also  point  toward  the  need  for  expanding  our  decision  to  accomodate  other  perspectives  such  as  interpretive 
and  critical  forms  of  inquiry,  as  well  as  the  empirical. 

A way  to  expand  my  discussion  to  accomodate  other  perspectives  that  can  be  used  in  looking  at  the  relationship 
between  philosophy,  inquiry  and  education  can  be  found  in  examining  research  paradigms.  This  framework  (research 
paradigms)  extends  the  more  traditional  descriptive,  normative  and  analytic  perspectives  by  looking  at  methodological/ 
epistemological  viewpoints.  This  framework  allows  the  researcher  to  identify  human  interests  within  modes  of  inquiry. 

Bredo  and  Feinberg  (1982)  discuss  differing  paradigms  according  to  the  research  methodologies  utilized.  These 
methodologies  have  inherent  interests  in  the  kind  of  research  findings  sought  and  generated.  The  paradigms  identified  are 
the  empirical,  or  positivistic,  the  interpre  tive,  and  the  critical  approaches  to  social  and  educational  research.  These 
paradigms  have  fundamental  differences  that  separate  the  positivistic  from  the  interpretive  and  critical  approaches.  These 
differences  are  of  a philosophical  nature  concerning  the  nature  of  reality  (metaphysics),  subject-object  dualism, 
generalization,  causality,  and  axiology  (Koetting,  1985;  1993),  and  how  we  come  to  know  reality  in  any  way  whatsoever 
(epistemology).  These  concerns  keep  us  rooted  in  doing  philosophy. 


Although  I have  used  the  term  "foundations"  frequently  in  this  section,  I am  not  talking  about  the  establishment 
of  a "meta-narrative"  (Lyotard,  1984;  Hlynka  and  Yeaman,  1992).  I am  not  talking  about  "doing  philosophy  in  the  grand 
manner"  of  building  systems  of  thought  (Wingo,  1974).  I do  not  believe  that  there  is  only  one  complete  explanation  or 
understanding  of  our  social  world  and  that  given  the  time  and  effort  we  will  be  able  to  "figure  things  out".  What  I am 
saying  about  foundations  is  the  way  in  which  philosophy  is  carried  out,  within  philosophical  inquiry,  the  problem- 
posing,  the  questioning,  the  search  for  clarification,  the  quest  for  seeing  things  relational^,  provides  multiple  ways  of 
inquiring  into  the  world  of  social  and  educational  realities. 

Martusewicz  and  Reynolds  (1994)  state  a similar  position  regarding  foundations.  They  see  the  purpose  of 
foundations  to  raise  questions  and  offer  points  of  view  that  ask  us  to  see  what  we  do  as  teachers  or  as  students  in  new  or 
at  least  unfamiliar  ways,  from  another  side,  perhaps  from  the  inside,  of  perhaps  from  both  inside  and  outside.  It  is  an 
invitation  to  look  at  education  both  socially  and  historically  as  well  as  practically,  that  is,  from  the  inside  (the  complex 
processes,  methods,  and  relations  that  affect  individuals  in  schools,  for  example)  within  the  context  of  the  outside  (the 
larger  social,  economic,  and  political  forces  that  have  affected  those  processes  over  time\  (p.2) 

This  notion  of  inside/outside  (school/world)  provides  a sociocultural  perspective  that  suggests  the  "flux  of 
boundaries",  and  allows  the  researcher/participant  to  see  the  relationships  of  seemingly  separate  realities,  as  well  as 
questions  the  idea  of  foundations  as  a "stable  set  of  knowledges,  concepts,  or  principles  to  be  discovered,  defined,  and  then 
presented  in  a unilinear  way"  (Martusewicz  and  Reynolds,  1994,  p.3;  also  see  Greene,  1974).  Again,  there  is  no  one  best 
way  to  explain  what  happens  in  the  world  or  in  education.  Possibilities  for  understanding,  however,  can  take  place  when 
we  pay  "particular  attention  to  perspectives  that  maintain  a critical  stance,  a willingness  to  put  existing  assumptions  and 
interpretations  into  question"  (Martusewicz  and  Reynolds,  1994,  p.3). 

As  I become  engaged  in  this  form  of  critical  inquiry,  I become  involved  in  theory  (foundations).  There  are 
multiple  theoretical  perspectives  on  the  world,  knowledge,  value.  However,  there  is  no  meta- narrative.  There  is  no  grand 
philosophy  (Greene,  1974;  1994;  and  Martusewicz  & Reynolds,  1994).  Returning  to  an  assumption. made  earlier  from 
Wingo  (1974),  'l 

Education  always  takes  place  within  a certain  constellation  of  cultural  conditions  and  therefore  it  cannot  be 
studied  as  a set  of  universal  and  independent  phenomena.  Some  set  of  relations  among  education,  politics,  and  social 
institutions  is  inevitable  and  cannot  be  ignored  in  any  useful  analysis  (p.24).  There  are  multiple 
explanations/understandings  of  schooling,  of  the  "world",  and  multiple  ways  of  knowing.  Multiple  ways  of 
understanding  the  world  of  education  have  to  do  with  theory.  Understanding  the  notion  of  theory  can  provide  insight  into 
the  multiple  interpretations  of  the  world  and  experience.  Theorizing  is  a mode  of  philosophical  inquiry  that  suggests  the 
complexities  and  possibilities  for  creating/constructing  knowledge.  And  that  is  the  subject  of  another  scries  of  papers.3 
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This  research  investigated  how  use  of  an  interactive  videodisk  information  system  helped  preservice  teachers  (PSTs) 
expand  their  visions  of  teaching,  learning,  and  assessment  in  mathematics  and  their  skills  in  translating  that  vision  into 
action  in  the  classroom.  The  technology  was  used  as  one  element  in  the  field  experience  component  of  a masters 

the  technology  in  formulating  personal  visions  of  effective  teaching,  how  they  planned  and  taught  lessons  to  actual 
classes  of  children,  how  they  critiqued  their  own  teaching  and  the  teaching  of  others,  and  how  their  philosophies  and 
beliefs  about  teaching  and  learning  developed  throughout  the  course. 

Rationale 

Changes  are  in  the  air  in  schools  in  the  United  States:  changes  in  curricula,  changes  in  teaching  methods,  and 
changes  in  assessment.  As  a result,  teacher  education  must  change  as  well.  In  March  1989,  the  National  Council  of 
Teachers  of  Mathematics  (NCTM)  released  its  Curriculum  and  Evaluation  Standards  for  School  Mathematics.  Central  to 
the  message  of  the  Curriculum  and  Evaluation  Standards  is  the  development  of  mathematical  power  for  all  students, 
described  as  the  ability  "to  explore,  conjecture,  and  reason  logically  as  well  as  the  ability  to  use  a variety  of  mathematical 
methods  effectively  to  solve  nonroutine  problems.  This  notion  is  based  on  the  recognition  of  mathematics  as  more  than 
a collection  of  concepts  and  skills  to  be  mastered;  it  includes  methods  of  investigating  and  reasoning,  means  of 
communication,  and  notions  of  context"  (NCTM,  1989,  p.  5). 

Research  findings  from  cognitive  psychology  and  mathematics  education  indicate  that  students  learn  as  they 
actively  assimilate  new  information  and  experiences  to  construct  their  own  meanings  (Case  & Bereiter,  1 984;  Cobb  & 
Steffe  1983;  David,  1984;  Hiebert,  1986;  Lampert,  1986,  1990;  Schoenfeld,  1987).  Since  this  is— of  course— just  as 
true  of  the  learning  of  prospective  teachers  as  of  any  students,  we  as  teacher  educators  must  practice  what  we  teach.  That 
is,  we  cannot  hope  to  produce  teachers  who  can  adapt  to  new  notions  of  what  it  means  to  teach  mathematics  unless  we 
change  our  own  teaching  practices  as  well. 

Recognizing  that  reaching  the  goal  of  developing  mathematical  power  for  all  students  requires  teachers  with  new 
and  different  philosophies  and  competencies,  the  NCTM  published— in  1 991— a companion  set  of  standards:  the 
Professional  Teaching  Standards  for  School  Mathematics.  This  document  identifies  the  need  for  mathematics  teachers 
who  are  proficient — among  other  things — in  "selecting  mathematical  tasks  to  engage  students’  interests  and  intellect; 
providing  opportunities  to  deepen  students'  understanding  of  the  mathematics  being  studied  and  its  applications; 
orchestrating  classroom  discourse  in  ways  that  promote  the  investigation  and  growth  of  mathematical  ideas;  using,  and 
helping  students  use,  technology  and  other  tools  to  pursue  mathematical  investigations;  seeking,  and  helping  students 
seek,  connections  to  previous  and  developing  knowledge;  [and]  guiding  individual,  small-group,  and  whole-class  work" 
(NCTM,  1991,  p.  1). 

The  goal  of  this  research  study  was  to  investigate  how  use  of  an  interactive  videodisk  information  system  (the 
Strategic  Teaching  Framework)  in  a field  experience  course  for  preservice  teachers  (PSTs)  might  help  them  expand  their 
visions  of  teaching,  learning,  and  assessment  in  mathematics  and  their  skills  in  translating  that  vision  into  action  in  the 
classroom. 


certification  program  for  prospective  elementary  teachers  and  provided  them  with  models  of  effective  teaching  around 
which  class  discussions  and  individual  journal  reflections  were  based.  The  data  we  collected  document  how  students  u 


The  Strategic  Teaching  Framework 

Strategic  Teaching  Framework9  (STF)  is  an  interactive  videodisk  information  system  designed  to  help 
individuals  develop  teaching  strategies  that  are  problem-solving  oriented,  that  focus  on  both  students  and  processes,  and 
that  utilize  collaborative  work  as  a means  for  testing  ideas.  The  system  is  written  in  HyperCard  and  runs  on  a Macintosh 
computer.  Videos  of  classrooms  are  stored  on  videodisk  with  presentation  of  the  video  and  supporting  materials 
controlled  through  the  STF  interface. 

STF  was  designed  to  support  teachers  through  a constructivist  or  process-oriented  learning  environment  (Duffy, 
in  press) — in  particular,  the  sort  of  environment  reflected  in  models  of  cognitive  apprenticeship  (Brown,  Collins,  & 
Duguid,  1989)  and  of  the  reflective  practitioner  (Schon,  1989).  The  STF  module  consists  of  video  of  a full  class  period 


9 Strategic  Teaching  Framework  was  a joint,  project  of  Indiana  University  and  the  North  Central  Regional  Education 
Laboratory.  Lead  designers  for  the  program  were  Thomas  Duffy,  Beau  Jones,  and  Randy  Knuth. 
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of  teaching  by  three  elementary  school  teachers10,  each  of  whom  exhibits  characteristics  consistent  with  the 
recommendations  of  the  Professional  Teaching  Standards  (NCTM,  1991).  One  of  the  STF  design  strategies  was  to 
provide  access  to  multiple,  sometimes  very  different,  representations  of  exemplary  (learner-centered,  collaborative, 
problem  driven)  practice  to  aid  prospective  teachers  in  understanding  that  the  teaching  strategies  illustrated  should  not  be 
seen  as  practices  to  be  slavishly  imitated  but,  rather,  as  providing  the  underpinnings  for  a conceptual  framework  to  guide 
practice. 

The  videos  of  the  three  teachers  each  teaching  a full  class  period  of  instruction  is  the  heart  of  the  STF  system. 
STF  very  consciously  does  not  focus  on  isolated  skills  or  on  idealized  demonstrations,  but  instead  emphasizes  the 
interrelatedness  of  teaching  practices  by  presenting  them  in  context.  The  design  strategy  was  to  avoid  teaching  the 
critical  attributes  in  the  video  for  two  reasons.  First,  the  critical  points  or  attributes  in  an  actual  teaching  experience  are 
really  a matter  of  one's  perspective  rather  than  a reality.  Second,  within  the  constructivist  and  apprenticeship  mode,  the 
goal  was  to  aid  the  learner  in  developing  and  evaluating  his  or  her  own  perspective  on  critical  issues.  It  was  believed  that 
learning  would  be  more  effective  if  it  was  learner  (teacher)  centered  rather  than  system  specified.  The  perspectives 
provided  on  the  teaching  in  the  videos  further  reflected  this  design  strategy  of  supporting  the  teacher-learner’s  constructive 
activity  and  providing  a basis  for  testing  those  constructions.  Rather  than  a single  perspective  on  the  important  aspects 
of  the  teaching,  STF  provides  commentary  from  three  different  perspectives:  the  featured  teacher,  a mathematics  educator, 
and  the  STF  developers.  The  perspectives  are  ’’attached"  to  the  video,  so  that  the  learner  can  obtain  the  three  perspectives 
on  the  palicular  teaching  activity  currently  being  viewed. 

A second  part  of  the  STF  information  system  is  a conceptually  organized  information  base.  Topics  include 
assessment,  management,  teaching  strategies,  problem  solving,  and  planning.  For  ear',  of  these  topics  there  are  4 to  12 
subtopics,  and  for  each  subtopic  the  learner  can  get  several  types  of  information:  ^ , i pectives  on  the  video  and  the 
videoclip  that  are  directly  related  to  that  subtopic;  references  to  a specific  reference  article  in  a hardcopy  library  that  is  part 
of  STF;  or  an  interview  with  the  teacher  or  with  one  of  three  education  experts.  Thus,  after  studying  the  classroom  as  a 
whole,  the  learner  can  move  to  the  information  base  to  develop  a richer  understanding  of  various  aspects  of  the  teaching. 
Additionally,  a notetaking  feature  is  built  in  to  the  system  to  allow  users  to  attach  their  own  comments  to  or  establish  a 
dialogue  around  a video  segment. 


The  Research  Agenda 

The  focus  of  this  research  project  was  the  design  and  formative  evaluation  of  an  instructional  model  to  support 
the  use  of  STF  in  mathematics  teacher  education.  Because  this  was  a first  evaluation  of  the  STF  system  in  a teaching 
context,  it  was  important  to  document  in  considerable  detail  how  the  students  worked  with  the  STF  system  and  how  they 
implemented — in  their  field  experience  classrooms — the  strategies  illustrated  by  STF.  We  also  were  interested  in 
describing  the  extent  to  which  the  PSTs  exhibited  reflective,  situated  decision  making  during  teaching.  Data  on  use  of 
*,STF  came  from  observations,  student  journals,  and  computer  logs  of  the  frequency  and  types  of  use.  (These  data  will  also 
serve  later  to  guide  revision  of  the  system.) 

Our  instructional  model  emphasized  PSTs’  construction  and  testing  (through  collaboration,  observation,  and 
practice)  of  their  own  conceptual  frameworks  for  what  it  means  to  teach  mathematics  according  to  the  recommendations 
of  the  NCTM  standards  documents.  It  also  encouraged  the  prospective  teachers  to  reflect  both  on  teaching  practices  and  on 
students’  learning — a promising  strategy  for  changing  the  way  mathematics  is  taught,  learned,  and  assessed  (Tobin, 

1989).  When  PSTs  actively  participate  in  evaluating  classroom  lessons,  they  can  begin  to  create  new,  more  effective 
frameworks  for  helping  the  students  they  teach  make  mathematical  connections. 

Our  instructional  model  contained  elements  of  both  the  cognitive  apprentice  and  the  reflective  practitioner 
models  of  teacher  education  (Duffy,  in  press).  Central  to  both  of  these  models  is  the  notion  that  learning  must  be 
purposeful — that  there  should  be  a realistic  task  or  goal  for  which  the  learning  will  be  used.  This  notion  provides  learners 
with  a framework  for  developing  self  regulation  in  identifying  learning  goals,  carrying  out  designated  tasks,  and  self- 
assessing  progress  in  learning  through  accomplishing  the  task.  In  our  model,  the  task  was  for  each  PST  to  develop  and 
deliver  mathematics  instruction  to  the  class  of  elementary  students  to  which  he  or  she  had  been  assigned  for  a semester- 
long  early  field  experience  in  mathematics  and  science  teaching. 

Consistent  with  the  cognitive  apprenticeship  model,  STF  provides — through  its  video  exemplars,  its  clips  of 
expert  commentary,  and  its  library  of  resources — expert  models  and  multiple  perspectives  on  their  performance.  Further, 

10  These  exemplary  teachers  are  Vickie  Bill,  St.  Agnes  Elementary  School,  Pittsburgh;  David  Birchfield,  Brownsville 
Elementary  School,  Charlottesville  VA;  and  Dwight  Cooley,  Alice  Carlson  Applied  Learning  Center,  Ft.  Worth  TX.  At  the 
time  of  this  study,  Dwight  Cooley  had  not  yet  been  added  to  the  STF  system.  His  class  was  only  available  on  videotape. 
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collaboration  among  peers,  as  well  as  interactions  with  the  university  supervisor,  the  cooperating  teachers,  and  the 
elementary  students  in  the  classrooms  provided  a vehicle  for  the  PSTs  to  develop  and  test  their  own  conceptual 
frameworks  for  instruction.  Consistent  with  the  reflective  practitioner  model,  our  use  of  STF  emphasized  a cycle  of 
reflection-in-action.  This  reflection-in-action  began  with  the  PSTs  reflecting  on  the  action  of  two  teachers  in  the  STF 
video,  moved  next  to  reflections  on  the  teaching  of  the  cooperating  teacher  in  the  field  experience  classroom,  and 
progressed  to  reflection  on  the  PST's  own  performance  in  the  classroom. 

This  paper  focuses  specifically  on  the  use  and  impact  of  STF  in  a course  for  PSTs,  though  the  work  reported 
here  represents  just  a portion  of  a larger  study  designed  to  examine  the  PSTs  growth  in  thought  and  action  and 
development  as  reflective  practioners. 


Method 

Subjects 

The  subjects  in  the  study  were  16  preservice  teachers  enrolled  in  Indiana  University's  Elementary  Certification 
Graduate  Program  (ECGP).  Students  in  the  class  of  sixteen  ranged  from  23  - 45  years  of  age,.  Only  one  student  (Pete11) 
was  already  a certified  teacher  (of  secondary  social  studies),  but  several  students  had  experience  as  substitute  teachers, 
tutors,  or  classroom  volunteers.  Roughly  half  the  class  was  married,  arid  many  had  children  at  home.  All  students  were 
changing  current  careers  to  become  elementary  teachers.  The  class  was  composed  of  ten  women  and  six  men. 

All  of  these  students  contributed  to  the  data  on  the  use  of  STF.  Eight  of  them,  four  men  and  four  women,  were 
selected  for  a more  intensive  analysis  of  their  journals  and  other  reflective  contributions  regarding  their  beliefs,  attitudes 
and  strategies  for  using  STF  and  its  contribution  to  their  growth  as  a perspective  teacher.  These  students  were  selected  at 
the  end  of  data  collection,  but  prior  to  any  analysis,  based  on  two  criteria.  First,  they  had  written  extensive  journals. 
Second,  in  the  teachers  estimation  and  based  on  class  assessment  instruments,  these  eight  students  seemed  to  represent  a 
r*  cross  section  of  the  class  in  terms  of  range  of  their  understanding  of  learner  centered  instruction,  their  classroom 
experiences,  and  their  growth  over  the  course  of  the  semester. 

The  Teacher  Education  Program 

The  ECGP  program  is  designed  for  students  who  already  have  liberal  arts,  humanities,  or  social  or  natural 
science  baccalaureate  degrees,  but  have  returned  to  the  university  to  work  concurrently  on  certification  as  elementary 
teachers  and  a masters  degree.  The  program  seemed  a particularly  appropriate  site  for  experimenting  with  use  of  STF  in  a 
teacher  education  setting  because  the  ECGP  program  was  explicitly  founded  upon  the  following  principle:  "Teacher 
education  is  viewed  primarily  as  a process  of  praxis.  Students  are  assisted  to  construct  their  own  personal  theories  of 
education  (based  upon  personal  experiences  and  course  experiences),  are  given  opportunities  to  test  these  theories  in  a 
variety  of  field  experiences,  and  are  continually  encouraged  to  reflect  upon  all  three  types  of  experiences  (personal,  course, 
and  field)  in  revising  and  reconceptualizing  their  theories"  (ECGP  Program  Description,  1993,  p.  3). 

The  Curriculum  and  Course 

The  students  were  concurrently  enrolled  in  E543  (Advanced  Study  in  the  Teaching  of  Mathematics  in  the 
Elementary  School)  and  M501  (Laboratory /Field  Experience,),  both  taught  by  the  first  author.  Use  of  STF  was  a new, 
but  integral,  part  of  the  ECGP  M501  field  experience. 

M501  is  an  early  field  experience  course  scheduled  to  meet  one  full  day  per  week  for  a semester.  As  part  of  the, 
course,  each  prospective  teacher  is  assigned  to  an  elementary  school  classroom  and  spends  one  day  per  week  for  about  12 
weeks  at  the  school  with  that  teacher  and  his  or  her  class.  The  PSTs  enrolled  in  M501  are  given  four  mathematics 
assignments  to  be  carried  out  in  the  school:  (1)  to  conduct  a series  of  interviews  with  three  children — at  diverse  levels  of 
ability  — in  an  elementary  school  classroom,  and  (2)  - (4)  to  design  and  present  three  mathematics  lessons  to  the  class  as 
a whole. 

Before  developing  their  interview  questions,  the  PSTs  view  a videotape  on  effective,  interviewing  techniques  for 
mathematics  assessment  (Burns,  1993).  They  then  conduct  their  interviews  with  children,  audio-recording  one  of  them 
for  later  review,  transcription,  and  analysis  in  a paper  written  for  the  course.  The  interview  assignment  serves  two 
purposes:  to  provide  the  PSTs  with  a detailed  look  at  how  children  of  a particular  age  think  about  mathematical  concepts 
and  solve  problems,  and  to  encourage  them  to  reflect  on  their  own  ideas  about  children's  understandings  and  their  own 
questioning  techniques. 


1 1 Names  used  are  psueudonyms  for  students  in  the  study. 


In  each  of  the  three  lessons  that  the  PSTs  design  and  teach  (two  of  which  are  eventually  self-critiqued  in  course 
papers),  they  are  instructed  to  incorporate  at  least  one  of  the  following:  collaborative  learning,  the  use  of  manipulatives, 
teaching  a mathematical  concept  via  problem  solving,  and  deriving  mathematical  problems  from  narratives  (i.e., 
literature-based  mathematics  teaching).  These  aspects  of  mathematics  instruction  are  supported  by  recommendations  of 
the  NCTM  Standards  and  are  also  illustrated  in  the  STF  module. 

Incorporating  the  use  of  STF  into  the  M501  curriculum  required  a reallocation  of  time  during  the  course.  For 
weeks  two  to  five  of  the  term,  students  spent  half  of  their  M501  course  time  (that  is,  one  afternoon  per  week)  at  the 
university  working  with  STF  and  on  the  design  of  lesson  plans.  During  these  same  weeks,  the  other  half  of  their  M501 
course  time,  (that  is,  one  morning  per  week)  was  spent  either  in  teaching  workshops  (during  weeks  two  and  three)  or  in 
work  in  their  field  experience  classroom  at  the  school  (during  weeks  four  and  five).  From  the  6th  to  the  1 3th  week  of  the 
term,  the  PSTs  spent  an  entire  day  each  week  in  their  field  experience  classrooms,  but  had  a one-hour  meeting  per  week 
at  the  university  for  discussion  and  debriefing  (as  well  as  three  hours  per  week  in  Mathematics  Methods  class  — where 
considerable  discussion  about  teaching  took  place  as  well).  The  PSTs'  teaching  of  mathematics  lessons  in  the  schools 
was  done  during  weeks  6-9  of  the  semester. 

The  resources  available  to  the  prospective  teachers  as  they  planned  their  lessons  included: 

• The  classroom  to  which  they  were  assigned  (as  a source  of  information  on  the  culture  of  the  classroom  and 
the  skills/knowledge/problem  solving  strategies  of  the  children). 

• The  mathematics  methods  class  (as  a source  of  information  on  appropriate  content  and  sequencing  for  math 
instruction). 

• STF  (as  a source  of  information  on  mathematics  teaching  strategies  and  goals). 

STF  Facilities 

Four  Apple  llci  workstations  were  available  for  the  class.  The  peripherals  associated  with  each  computer 
included  a CD-ROM  player,  video  disk  player,  and  monitor.  Copies  of  the  STF  software  and  videodisks  were  kept  with 
each  workstation.  Students  were  also  able  to  check  out  videotapes  and  hardcopies  of  the  CD-ROM  information  for  home 
use. 

The  computers  were  in  laboratory  settings  which  included  a large  table  and  space  for  the  team  to  gather  and 
work.  Other  students  in  one  of  the  School's  programs  had  access  to  these  computers  for  their  work,  but  the  PSTs  had 
priority  and  were  able,  to  access  the  machines  any  time  they  desired.  The  shared  access,  however,  did  mean  that  some 
reconfiguring  was  often  necessary  before  the  students  could  work.  The  two  most  frequent  issues  were  disconnecting  the 
machines  from  the  network  and  reattaching  the  CD  ROM  player  as  a peripheral. 

Strategy  for  Using  STF 

There  were  STF  based  classroom  assignments  each  Monday  afternoon  for  weeks  2-5  of  the  class  (the  first  week 
was  spent  orienting  students  to  the  course).  For  this  paper,  these  weeks  will  be  referred  to  as  STF  week  1 (or  session  1), 
STF  week  2 (session  2),  etc..  Additionally  the  students  had  access  to  STF  at  any  time  throughout  the  semester.  The 
students  worked  with  STF  in  teams  of  four  to  five,  with  each  team  assigned  to  one  of  the  four  computers. 

In  the  first  STF  session,  the  M501  class  examined  the  teaching  of  one  of  the  three  teachers  in  the  STF  system 
(Vickie  Bill).  Each  group  viewed  the  video  of  Vickie  Bill’s  lesson  with  the  goal  of  identifying  the  key  features  of  her 
approach  to  teaching  and  analyzing  how  and  why  these  components  of  the  teaching  are  important.  These  analyses  were 
then  shared  in  the  final  hour  of  the  class,  in  whole-class  discussions  and  further  elaborated  upon  in  individual  journal 
assignments.  As  part  of  their  reflective  journals  (which  were  turned  in  weekly  for  commentary  by  the  first  and  third 
authors),  PSTs  recorded  their  thoughts  on  the  implications  of  their  analysis  of  the  STF  teachers  for  their  own  images  of 
effective  mathematics  teaching  and  for  their  own  soon-to-be  written  mathematics  lesson  plans. 

In  STF  week  2,  the  students  discussed  further  their  reaction  to  Vickie  Bill,  then  were  sent  in  their  groups  to 
watch  and  analyze  second  teacher  in  the  STF  system,  David  Birchfield.  As  with  Vicki,  they  were  asked  to  identify  the 
key  features  of  David's  teaching  and  analyze  how  and  why  these  components  were  important.  During  the  week,  students 
were  asked  to  continue  to  analyze  the  teaching  of  David  Birchfield,  focusing  on  questioning  strategies.  Discussion, 
analysis,  and  reflection  via  journal  ensued  similar  to  that  of  week  1,  with  additional  emphasis  on  analyzing  both  David 
and  Vickie's  questioning  strategies. 

In  STF  week  3,  students  were  sent  in  their  groups  to  the  STF  stations  immediately,  to  identify  and  prepare  to 
bring  back  to  the  class  examples  of  Vickie  and  David  which  exemplified  what  they  believed  to  be  key  elements  to 
teaching  (which  required  the  group  to  come  to  consensus  on  those  elements  first).  Students  spent  the  last  half  of  the 


class  presenting  their  STF  examples  and  engaging  in  discussion  about  them.  Journal  entries  for  this  week  focused  on 
their  questions,  concerns,  and  insecurities  about  their  field  experience  (they  had  visited  their  classrooms  for  the  first  time 
that  morning).  They  were  asked  to  use  STF  to  explore  and  comment  on  these  issues. 

The  fourth  week  of  STF  work  involved  students  in  actually  conceptualizing  and  writing  a plan  for  a 
mathematics  lesson.  This  planning  was  done  in  small  groups,  shared  and  critiqued  in  whole  class  discussion,  and  then 
became  the  topic  of  a reflective  writing  assignment  for  individuals' journals. 

Field  Experience 

The  PSTs  were  each  assigned  to  an  elementary  (grades  1 to  6)  classroom  and  teacher  in  one  of  two  schools 
within  a twenty  minute  drive  of  the  University.  Once  the  students  were  working  full-time  in  the  elementary  school 
(beginning  in  week  5),  they  proceeded  to  the  tasks  of  conducting  their  diagnostic  interviews  and  planning  and  teaching 
their  three  lessons.  Each  week  they  wrote  extensive  commentary  in  their  field  experience  journals  about  observations 
they  were  making  in  the  classroom,  issues  that  were  arising,  beliefs  that  were  being  challenged,  and  successes  and  failures 
that  they  were  experiencing.  The  students  were  encouraged  to  return,  whenever  it  seemed  useful,  to  STF  as  a resource. 

Between  weeks  six  and  nine  the  students  were  required  to  teach  three  mathematics  lessons  in  the  elementary 
classroom.  The  first  and  third  authors  attended  at  least  one  of  the  teaching  sessions  of  each  PST  and  provided  written 
feedback  to  the  students  afterward.  They  were  also  videotaped  during  at  least  one  of  the  three  lessons.  Videotaping  was 
done  by  students  in  another  program  in  the  school.  The  tape  was  given  to  the  PST  after  class  with  instructions  to  watch 
the  video,  analyze  the  teaching,  reflect  on  their  own  performance,  and  write  a thoughtful  critique. 


Data  Sources  and  Analysis 

Data  about  students'  use  of  and  attitudes  about  the  STF  system  were  collected  in  a number  of  ways.  Keystroke 
data,  student  journals,  and  reflections  on  videotapes  of  their  own  teaching  provided  the  researchers  with  information  about 
the  use  and  impact  of  STF  throughout  the  entire  semester.  Additionally,  the  teacher  of  the  class  (the  first  author) 
provided  reflections  of  the  use  and  impact  of  STF  from  the  instructor’s  perspective. 

Keystroke  data.  STF  software  included  a program  to  record  login  information  and  the  time  of  all  keystrokes 
while  using  the  program.  This  data  provided  us  with  a record  of  what  STF  features  were  used  by  whom,  when,  and  for 
how  long. 

Student  reflective  journals  The  students  kept  weekly  journals  in  which  they  reflected  on  their  use  of  the  STF 
system,  their  growing  understanding  of  effective  teaching,  and  their  reactions  to  their  field  experiences  and  class  activities. 
The  first  and  third  authors  responded  in  writing  to  these  journal  entries  weekly,  creating  a written  conversational  trail  of 
students'  progress  in  thinking  and  researcher  reaction  to  it  throughout  the  course.  While  students  were  encouraged  to 
write  about  any  experience  or  reflection,  there  were  specific  questions,  topics,  or  issues  identified  for  journaling  each 
week.  These  questions  were  based  on  a combination  of  the  goals  of  the  week's  activity  and  the  nature  of  the  discussion 
during  the  class  period.  The  journaling  questions  posed  for  each  of  the  four  weeks  STF  was  assigned  were  as  follows: 

Watch  and  analyze  Vickie  Bill  on  STF.  Based  on  what  you  see,  what  are  three  things  you 
believe  contribute  to  good  mathematics  teaching? 

Evaluate  STF  — what  do  you  think  is  helpful?  Any  complaints? 

Last  week  you  wrote  about  three  things  you  believe  contribute  to  good  mathematics 
teaching.  Since  then  you  have  watched  David  Birchfield  on  STF.  As  a result  of  that 
additional  experience,  what  would  you  like  to  add  to  or  change  in  your  previous  journal 
entry?  Explain  why. 

Discuss  and  comment  on  the  questioning  strategies  used  by  Vickie  Bill  and  David 
Birchfield.  Explain  why  you  think  their  strategies  are  effective  or  ineffective. 

Identify  the  two  aspects  of  STF  as  a technology  that  have  been  most  useful  to  you. 
Explain.  Identify  the  two  aspects  that  have  been  most  confusing  or  frustrating.  Explain. 
Overall,  how  useful  has  STF  been  in  helping  you  develop  notions  of  good  mathematics 
teaching  (on  a scale  of  1 = not  at  all  helpful  to  10=extremely  helpful)?  Explain. 

What  are  your  questions,  concerns,  and  insecurities  about  your  field  experience.  How  do 
you  plan  on  approaching  these  concerns?  Use  STF  to  explore  and  comment  on  these 
issues. 


STF  week  1:  1) 

2) 

STF  week  2:  1) 

2) 

3) 

STF  week  3: 


STF  week  4: 


1)  Discuss  the  group  lesson  planning  experience  on  Sept  26.  Was  it  helpful?  Could  it  be 
improved? 

2)  Discuss  your  field  experience  and  any  issues  that  it  is  raising  for  you. 

End  of  course  Students  were  asked  to  reflect  on  their  STF  experience  at  the  the  end  of  the  course  in  a 

journal  assignment  and  in  the  course  evaluation. 

The  journal  entries  for  the  entire  semester  were  analyzed  for  the  eight  case  study  students  in  the  class  (see 
“Subjects")  using  the  Constant  Comparison  Method  outlined  by  Glasser  and  Strauss  (1975).  The  entries  were  divided 
into  idea  units,  defined  as  a focused  comment  about  some  event  or  experience.  The  first  and  third  authors  worked  on 
successive  samples  of  journals  until  they  established  80%  agreement  on  the  division  into  idea  units  and  they  could  apply 
those  divisions  to  another  sample  with  similar  agreement.  Then  the  third  author  divided  the  remainder  of  the  journals 
into  idea  units. 

The  idea  units  were  sorted  by  each  of  the  researchers  into  categories  of  ideas  that  seemed  to  go  together.  The 
categories  were  then  named.  Several  iterations  of  this  process  led  to  a matrix  of  categories  into  which  comments  about 
experiences  were  placed.  Idea  units  were  categorized  on  two  dimensions:  the  experience  that  was  being  reacted  to;  and  the 
type  of  comment  about  that  experience.  The  experiences  being  reacted  to  are  the  primary  instructional  components  of  the 
course  plus  prior  experience  as  learners  and  teachers:  STF,  the  field  experience,  their  teaching  experience,  the  interview 
assignment,  activities  in  the  Math  Methods  class,  other  ECGP  classes,  and  experiences  outside  of  the  University 
program. 

The  comments  on  each  of  those  experiences  were  first  sorted  into  two  categories:  comments  that  discussed  what 
was  learned  from  the  experience;  and  comments  on  the  design  and  effectiveness  of  the  experience  as  part  of  the  course, 
e.g.,  how  the  use  of  STF  could  be  made  a more  effective  part  of  the  course.  This  classification  scheme  is  represented  in 
the  table  below  (see  Table  1).  Some  comments  were  also  cross-categorized  in  terms  of  how  the  experience  helped  them 
develop  new  insights  on:  teaching  practices;  learning  and  schooling;  specific  teaching  strategies  (e.g.,  the  strategy  for 
questioning);  guiding  their  personal  development  as  a teacher;  their  current  teaching  style/beliefs;  and  mathematics  as  a 
discipline. 

Table  1 

Data  Categorization  Scheme 


Comentary  on/ 
Learned  From 

Evaluation  of 

STF 

Field  Experience 

Teaching  (including  video  of 
them  teaching) 

Interviewing  children 
(including  video  on 
interviewing  skills) 

Instructor’s  Activities  . 

Other  EGCP  Classes 

Non-University  Experience 

Once  the  categorization  scheme  was  in  place,  the  first  and  third  author  independently  categorized  a random 
selection  of  13%  of  the  comments  to  establish  reliability.  This  process  yielded  80%  agreement.  Considering  the 
substantial  number  of  categories  and  the  placement  of  comments  into  multiple  categories,  we  believe  80%  agreement  is 
more  than  acceptable.  This  complete  analysis  is  supporting  a larger  research  effort.  For  purposes  of  the  research  reported 
here,  our  focus  is  on  the  nature  of  the  comments  about  STF. 

Teaching  videos  and  reflections  on  teaching.  During  the  last  two  weeks  of  class,  volunteer  students  (twelve  in 
all,  including  seven  of  the  eight  case  study  students)  brought  in  one  of  their  own  videotaped  lessons  and  reflected  on  their 
own  teaching  with  the  third  author.  Students  were  told  to  be  prepared  to  show  and  critique  examples  of  questioning, 
problem  solving,  and  group  work.  These  discussions/interviews  were  audio  taped,  transcribed,  and  unitized  in  the  same 
manner  as  the  journal  comments. 
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Tparhpr  Reflections  Towards  the  end  of  the  semester,  the  third  author  interviewed  the  instructor  (first  author) 
about  her  expectations  of  the  system,  the  use  of  STF  in  her  class,  the  resulting  impact  of  its  use  during  the  semester,  and 
ways  in  which  its  use  could  be  modified  for  future  classes.  The  interview  was  audiotaped  and  transcribed. 


Results  and  Discussion 

The  use  of  STF 

Twenty-nine  sessions  on  the  STF  system  were  recorded  with  an  average  overall  length  of  58.37  min.  per 
session.  Students  used  the  system  outside  of  the  normal  class  time  more  often  than  they  used  it  during  class:  13  in  class 
sessions  vs.  16  out  of  class  sessions.  Generally  all  four  members  of  the  team  participated  in  the  sessions.  However,  the 
sessions  outside  of  the  classroom  did  vary  in  group  size  with  at  least  one  of  the  sessions  involving  one  student  working 
with  STF  on  her  own. 

All  twenty-nine  sessions  occurred  during  the  four  weeks  that  STF  was  an  assignment  in  the  course.  Although 
the  students  used  STF  outside  of  classtime  and  were  enthusiastic  in  their  assessment  of  it,  they  did  not  use  STF  from 
weeks  six  to  thirteen  as  an  explicit  tool  for  preparing  the  lessons  they  taught  in  the  school  or  for  reflecting  on  their 
teaching  afterwards.  Most  students  felt  there  was  only  so  much  they  could  learn  ffom'the  two  teachers  in  STF  and 
indicated  in  their  journals  that  they  wished  they  had  more  STF  teachers  to  view.  Their  choice  not  to  return  to  STF  after 
the  first  month  most  likely  represents  this  saturation. 

Table  2 summarizes  the  mean  amount  of  time  spent  on  each  of  the  components  of  STF  and  the  proportion  of 
the  total  time  that  it  represented.  We  were  surprised  to  find  that  the  students  distributed  their  time  rather  evenly  over  the 
three  primary  information  resources:  the  video  (26%),  the  commentary  on  the  video  (21%),  and  the  database  of 
information  on  the  teaching  strategies  (26%)  represented  in  STF.  The  use  of  these  three  components  represented  70%  of 
the  time  spent  on  STF. 


Table  2 

Distribution  of  time  spent  on  the  components  of  STF 


Activity 

view 

video 

listen  to 
perspectives 

use 

database 

view 

elaboration 

listen  to 
orientation 

use 

notes 

Totals 

Mean  Time 

15.35 

12.37 

15.00 

1.80 

1.19 

4.10 

49.81 

Proportion  of 
Total  Time 

26.3% 

21.2% 

25.7% 

3.1% 

2.0% 

7.0% 

85.3% 

There  were  some  differences  between  the  STF  sessions  that  occurred  during  the  :;gular  class  schedule  (classtime 
STF)  and  those  sessions  initiated  by  the  students  outside  of  the  scheduled  class  time  (after-class  STF).  The  after-class 
STF  sessions  on  average  were  shorter  in  duration  (53.3  and  67.2  minutes  respectively).  In  the  classtime  STF  sessions, 
the  focus  tended  to  be  on  viewing  and  discussing  the  videos,  with  this  activity  consuming  47.7%  of  the  session  as 
compared  to  only  12%  of  the  after-class  sessions).  In  contrast,  the  after-class  sessions  tended  to  focus  on  the  use  of  the 
information  base  with  36.2%  of  the  session  spent  in  the  database  as  compared  to  10.4%  of  classtime  sessions.  For  both 
classtime  and  after-class  sessions  the  students  spent  about  a fifth  of  the  session  (22.4%  and  20.3%  respectively)  listening 
to  and  discussing  the  expect  perspectives  with  a mean  of  5.5  and  6.2  perspectives  respectively  listened  to  during  a 
session. 

There  had  been  some  concern  that  students  would  simply  view  the  videos  and  not  engage  in  any  analysis  beyond 
that  generated  from  their  observations.  Therefore,  we  were  pleased  to  see  that  all  three  of  the  primary  STF  information 
resources  — videos,  perspectives,  and  the  database  -were  used  about  evenly.  While  the  assignment  required  the  students 
to  analyze  the  videos,  that  analysis  did  not  specifically  require  viewing  anything  beyond  the  videos  themselves.  Hence, 
the  extensive  use  of  the  database  and  perspective  resources  does  suggest  that  the  students  found  this  supportive 
information  useful  in  their  analyses. 
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The  Ease  of  Use 

The  use  of  STF  was  not  without  technical  problems.  However,  most  of  the  problems  occurred  with  just  one  of 
the  four  stations.  There  were  persistent  problems  with  this  station  in  accessing  the  audio  and,  on  occasions,  it  was 
difficult  to  start  the  program.  The  cause  of  the  problems  was  never  identified.  These  problems  led  Gary,  one  of  the 
students,  to  say,  "Has  a day  yet  gone  by  when  all  four  STF  set-ups  were  all  working  properly  without  any  problems?  . . 

. If  so,  how  many  days  in  a row?"  The  only  other  equipment  difficulties  occurred  when  cables  connecting  various  pieces 
of  the  STF  system  were  switched  by  people  using  the  computers  for  non-STF  work,  including  connecting  the  computer 
to  the  LAN  (which  slowed  down  the  responsiveness  of  the  computer),  disconnecting  the  CD-ROM  player,  and 
disconnecting  the  videodisk  player.  The  students  generally  had  to  seek  help  to  reconnect  the  equipment. 

While  the  instructor  had  initial  concerns  regarding  the  ease  of  use  of  the  system,  looking  back  she  stated,  "My 
worry  about  the  students  being  able  to  handle  the  technology  probably  was  unfounded  ...  I think  some  of  [the  problems] 
— and  it's  not  the  system  at  all  — [were]  simply  our  own  physical  environment."  The  instructor  did  express  some 
concern  about  using  the  system  without  the  technical  support  she  received  during  this  project.  She  doesn’t  feel  confident 
that  she  could  troubleshoot  minor  hardware  problems  that  might  arise. 

STF,  itself,  was  easy  for  the  students  to  operate.  Even  novice  computer  users  felt  that  they  adapted  to  the 
system  quickly.  After  the  first  week  Matt  stated,  "I  have  enjoyed  using  the  STF  throughout  the  past  week.  Knowing  so 
little  about  cutting  edge  computer  technology,  it  was  a pleasant  surprise  to  be  working  with  something  that  is  so  logical 
and  user  friendly."  Students  had  few  complaints  about  the  STF  interface  and  few  suggestions  for  improvements.  Tina 
commented  that  she  disliked  having  to  turn  the  videodisk  over  as  often  as  she  did,  and  Matt  found  that  the  screen  saver  on 
his  workstation  was  distracting.  Additionally,  Pete  commented  that  he  wished  he  had  the  opportunity  to  see  the  entire 
lesson  without  a break  (the  videodisk  stops  at  natural  pauses  in  the  classroom,  thereby  breaking  the  lesson  into  many  1-5 
minute  segments  as  opposed  to  one  long  40  minute  video). 

The  usefulness  of  STF 

In  this  section,  we  report  the  students'  and  instructor’s  sense  of  the  usefulness  of  STF  as  a tool  for  learning. 
Their  comments  fell  into  two  broad  categories.  First,  there  were  comments  on  the  particular  components  or  features  of 
STF.  Second,  there  were  comments  on  the  placement  and  role  of  STF  in  the  curriculum  — its  perceived  usefulness  where 
it  was  placed  and  its  relation  to  other  components  of  the  curriculum. 

Value  of  specific  STF  features.  In  the  initial  design  of  STF,  there  was  some  discussion  of  the  value  of  placing 
the  teaching  video  on  videodisk.  We  wondered  if  students  could  be  equally  well  served  by  a lower-tech  version  of  the 
system  consisting  of  videotapes  and  hard  copy  printouts  of  the  commentary.  How  important  are  the  interactive  elements 
of  the  Hypercard/laserdisk  system?  The  Cognition  and  Technology  Group  (1992)  has  argued  for  the  value  of  laser  disk 
for  rapid  access  in  reviewing  and  discussing  the  vie!  os,  and  our  students  seem  to  agree.  Starting  with  the  fourth  week  of 
the  course,  we  made  available  videotape  copies  of  the  classrooms  along  with  hardcopy  transcripts  of  the  comments. 
However,  the  tapes  and  hardcopy  were  not  used,  even  while  the  students  were  still  making  extensive  use  of  the  computer 
based  system.  Indeed,  Pete  indicated  that  he  liked  the  control  provided  by  the  videodisk  player,  stating,  "I  like  being  able 
to  replay  what  they  did  and  hearing  several  different  explanations  for  why  they  do  certain  things,  It  really  gives  us  a 
chance  to  get  to  the  nuts  and  bolts  of  teaching.  It  is  one  thing  for  someone,  to  tell  us  how  to  do  it.  It  is  something  else 
to  see  someone  doing  it  in  a real  world  setting." 

Pete  also  noted  the  value  of  the  perspectives  on  the  teaching  that  were  provided:  "I  like  having  the  comments  of 
the  teacher  and  expert  commentary  right  there  with  the  video.  I get  to  hear  their  comments  while  what  happened  is  fresh 
in  my  mind.  I can  play  the  video  back  immediately  if  I don’t  know  what  the  comments  are  about."  Other  students 
echoed  similar  sentiments,  saying, 

It  is  also  beneficial  to  hear  what  teachers  have  to  say  about  their  own  classroom  practices.  It’s  nice  to  know 
firsthand  what  seems  to  work  and  what  doesn't.  The  comments  the  teachers  make  can  really  help  a prospective 
teacher  shape  their  own  beliefs  about  what  may  or  may  not  work  in  the  classroom.  (Donna) 

The  feature  I liked  the  most  were  the  on-line  comments,  assessments,  and  critiques  of  the  teacher.  They  are 
essential  in  giving  a context  to  what  is  being  viewed,  and  they  really  helped  me  to  focus  on  why  this  teacher 
was  better  than  most  and  the  things  that  she  did  that  were  most  (or  least)  effective.  (Sam) 

There  were  a large  number  of  comments  from  the  students  praising  STF  for  teaching  them  by  example  (and 
letting  them  decide  what  to  make  of  the  examples),  rather  than  by  telling  them  what  good  teaching  should  be.  In  effect, 
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they  told  us  that  they  recognized  STF  as  the  constructivist  environment  that  it  was  designed  to  be  and  appreciated  the  fact 
that  the  STF  experience  aligned  well  with  the  philosophy  of  the  rest  of  their  teacher  education  program.  Laura  put  it  this 
way, 


We  are  learning  in  our  classes  to  provide  students  with  as  many  experiences  as  we  can  to  help  them  develop 
strategies  that  are  best  for  them.  STF  provides  us  with  those  same  opportunities.  We  are  exposed  to  many 
different  teaching  methods.  By  having  these  different  experiences  to  learn  from,  I am  better  equipped  to  find 
what  method  is  better  for  me. 

Tina  explained  the  same  thing  somewhat  differently, 

I like  the  fact  that  taken  together  the  system  not  only  presents  a variety  of  places,  but  an  assortment  of  teaching 
styles  and  several  commentaries  on  the  advantages  and  disadvantages  of  each  one.  This  helps  you  understand 
that  good  teaching  is  often  a matter  of  opinion  and  you  need  to  figure  out  wh  J:  works  for  you.  And  it  helps  you 
do  that. 

In  a later  journal  entry,  Tina  elaborated  on  her  earlier  comments  about  the  importance  of  having  a variety  of 
exemplars: 

STF  is  helping  me  because  it  provides  a means  of  comparison  that  I might  not  have  had  otherwise.  I knew 
when  I first  saw  Vickie's  class  that  I did  not  like  it.  I did  not  know  why.  It  was  just  instinct.  It  just  did  not 
work.  But  after  watching  David  Birchfield  I could  see  why  I disapproved.  It  is  like  you  do  not  really  know  what 
"hot"  is  unless  you  have  experienced  cold  also.  STF  provides  both  and  thus  provides  me  with  a means  of  seeing 
more  clearly  what  I like  and  do  not  like  and  why. 

Matt  claimed  that  the  natural  anxiety  experienced  by  all  preservice  teachers  had  been  alleviated  somewhat  because  he  had 
seen  multiple  exemplars  of  good  teaching. 

Using  STF  has  given  me  another  frame  of  reference  to  draw  from  while  in  the  classroom;  because  of  that,  the 
level  of  anxiety  that  I am  now  experiencing  is  only  a fraction  of  what  it  might  have  been,  had  I not  experienced 
various  strategies  and  styles  via  STF. 

In  a later  journal  entry,  he  claimed  that  STF  had  also  helped  him  with  self  criticism. 

Being  able  to  watch  a larger  variety  of  teachers  has  also  helped  me  in  the  critique  of  my  own  ability.  For 
instance,  by  watching  a variety  of  teachers,  I have  found  that  I have  a broader  range  of  comparison,  both  for  my 
own  teaching  and  in  evaluating  their  teaching. 

However,  most  of  the  students  agreed  that  the  availability  of  more  exemplars  would  improve  the  STF  system 
considerably.  As  Tina  put  it, 

STF  is  really  good  resource,  but  without  more  teachers  and  lessons  on  it,  it  exhausts  its  possibilities  after  a 
while.  It  is  such  a good  idea  that  I hope  that  the  creators  will  add  a lot  more  examples  to  it  soon. 

The  instructor  also  indicated  that  the  use  of  more  exemplars  over  a wider  variety  of  grade  levels  would  be 
beneficial  to  students.  With  more  exemplars,  she  could  have  students  look  at  how  teachers  handle  different  mathematical 
concepts  such  as  fractions,  shapes,  etc..  With  only  three  lessons  on  the  system,  the  opportunity  to  use  it  to  support 
teaching  of  specific  concepts  is  extremely  limited. 

STF  as  a curriculum  component.  One  of  our  main  concerns  in  this  project  was  to  learn  how  STF  couL  best  be 
used  in  a preservice  teacher  education  course.  In  this  study,  STF  was  used  as  a "substitute"  for  field  work  at  the 
beginning  of  the  semester.  It  was  seen  as  a way  for  students  to  "visit"  models  of  teachers  incorporating  the  NCTM 
Standards  prior  to  being  immersed  in  the  field  experience.  One  set  of  curriculum  issues  centered  upon  the  placement  of 
STF  in  the  curriculum.  Here,  responses  were  gent  rally  positive  — we  seemed  to  have  gotten  it  mostly  right.  The 
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second  set  of  comments,  discussed  STF  relative  to  other  components  of  the  course.  Here  there  was  strong  sentiment  that 
STF  is  no  substitute  for  actual  classroom  experience.  The  students'  and  instructor's  commentary  related  to  these  issues 
follows. 

The  use  of  STF  at  the  beginning  of  the  course,  prior  to  the  students'  placement  in  schools,  was  pointed  out  by 
several  students  as  being  a very  appropriate  use  of  the  system.  As  Donna  stated. 

I’m  glad  we  were  able  to  view  Vickie  and  David  before  we  began  our  field  experiences  because  I think  they  gave 
us  a framework  of  reference,  or  ideas,  to  work  from.  While  we  must  each  adapt  our  own  teaching  style,  it  helps 
to  have  ideas  that  "work"  from  practicing  professionals. 

Other  students  also  recognized  that  the  early  introduction  to  Vickie  and  David  helped  to  establish  their  early 
views  about  teaching.  Sam  commented  that  "Vickie  and  David  helped  me  form  my  own  early  view  about  what  teachers 
do,  and  what  they  should  do.  I'd  never  seen  or  experienced  someone  teaching  like  either  one  of  them,  but  in  retrospect,  I 
really  wish  I'd  had." 

The  instructor  felt  that  the  students  "got  off  the  mark  a lot  faster  than  the  class  I taught  before"  in  regards  to 
their  understanding  and  reflections  on  mathematics  teaching  and  teaching  in  general.  She  stated,  "With  doing  workshops 
with  them  with  hands-on  materials  and  so  on,  so  that  by  the  end  of  two  weeks,  they  already  have  a totally  different 
picture  of  what  math  teaching  is  all  about . . . and  having  used  STF  . . . they  have  that  different  picture."  Despite  this, 
she  would  like  to  utilize  STF  throughout  the  semester.  Ideally,  she  would  like  to  "be  able  to  have  it  right  in  the 
classroom,  so  that  on  any  day  that  we  had  our  methods  class,  we  could  say,  'Let's  break  up  right  now  and  spend  15 
minutes  . . . 

While  most  students  appreciated  the  use  of  STF,  almost  all  believed  that  hands-on  time  in  the  classroom  was 
even  more  valuable  to  them.  Some  students  indicated  in  very  strong  terms  that  any  time  out  of  the  classroom  was 
wasted  time: 

Now  that  I think  about  all  the  time  we  spent  using  it,  we  could  have  spent  4 more  afternoons  in  the  classroom 
— that  would  have  been  12  more  hours  of  practical  classroom  experience. ...  I did  observe  teaching  and 
learning  through  STF,  but  I wasn't  there.  I couldn't  walk  around.  I couldn’t  really  see  what  all  of  the  students 
were  doing  and  I couldn't  ask  questions  of  the  teacher  or  students.  If  I had  a question  other  than  the  programmed 
responses  that  were  provided,  I was  out  of  luck. 

Gary  echoed  similar  sentiments: 

Anything  that  kept  us  out  of  our  classroom  was  less  valuable  than  the  classroom  experience  would  have  been 
and  therefore,  a sacrifice.  I did  enjoy  watching  the  two  STF  videos,  but  would  rather  have  done  that  in  one 
afternoon  instead  of  three. 

Jennifer  made  a similar  comment  about  the  number  of  examples  that  students  were  able  to  analyze  via  STF  vs. 
classroom  observation  during  the  same  time  frame:  "Instead  of  watching  those  two  [Vickie  and  David]  teach  the  same 
lesson  four  weeks  in  a row,  I would  much  rather  have  watched  my  teacher  teach  four  different  lessons."  However,  she 
goes  on  to  say,  "This  is  not  to  say,  however,  that  I did  not  get  any  ideas  from  STF  as  to  what  I would  like  and  would  not 
like  to  do  in  my  class  — I did." 

Other  students  recognized  that  while  nothing  replaced  actually  being  in  the  classroom,  STF  did  provide  a useful 
substitute.  As  one  student  stated,  "I  truly  believe  that  I need  to  be  in  the  classroom  to  achieve  full  understanding  of  what 
is  and  is  not  effective;  but  this  [STF]  is  a great  substitute." 

Gary  had  very  strong  sentiments  about  the  expenditure  of  resources  and  funds  used  to  create  system  of  this  type. 
He  commented,  "to  me,  STF  is  just  another  impediment  to  having  a fifteen-to-one  student-teacher  ratio."  He  later 
expanded  on  this  thought  by  stating: 

It's  not  that  I dislike  the  lessons  that  I got  from  the  STF,  they  were  certainly  useful.  I just  can't  help  thinking 
about  all  of  the  resources  that  were  tied  up  delivering  those  two  lessons.  I don't  think  it's  worthwhile.  If  I ask 
myself:  "how  much  better  of  a teacher  will  I be  for  having  been  exposed  to  STF?,"  in  all  honesty,  I have  to 
answer:  "not  that  much." 
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Use  of  STF  was  only  one  part  of  an  entire  course  designed  to  get  students  to  think  about  and  practice  teaching  in 
the  style  of  the  NCTM  Standards.  This  integration  and  focus  of  all  aspects  of  the  course  to  a common  goal  makes  the 
contribution  of  individual  components  difficult,  if  not  impossible  to  determine.  Two  of  the  eight  case  study  students 
recognized  and  commented  on  this  facet  as  well.  Laura  discussed  how  various  components  of  the  course  opened  up  her 
mind  to  new  ideas,  stating  "from  my  readings, . . . viewing  the  STF  videos  and  in-class  videos,  and  interviews,  I am 
beginning  to  give  kids  credit  and  keep  myself  open  because  I think  they  will  surprise  me  and  exceed  my  expectations." 
Reflecting  back  on  the  course  at  the  end  of  the  semester,  Matt  commented  on  the  integration  of  STF  with  the  other 
aspects  of  the  course  in  being  fundamental  in  his  growth: 

No  individual  experience  in  the  math  methods  course  would  have  been  complete  without  the  other  sections  to 
compliment  it,  and  in  that  sense  it  seems  to  me  that  all  sections  of  the  course  meshed  together  well  and 
developed  into  a final  product  that  was  very  conducive  to  understanding  of  mathematics  in  the  elementary 
classroom. 

How  STF  impacted  understanding, 

STF  teachers  as  mental  models.  One  of  the  primary  goals  of  the  course  instructor  was  to  provide  exemplars  of 
teaching  math  that  the  students  could  "see"  since  the  cooperating  teachers  they  were  placed  with  may  or  may  not  be 
teaching  in  the  spirit  of  the  NCTM  Standards.  She  felt  that  students  would  be  able  to  "get  a chance  to  do  more  than  just 
see  the  classroom,  but  that  they  could  sort  of  get  inside  the  teacher's  head,  or  get  inside  some  other  people's  heads  about 
what’s  going  on."  Student  comments  and  observation  of  students  in  the  field  indicate  that  the  teachers  in  the  STF  videos 
did  serve  as  mental  models  of  good  teaching  for  the  preservice  teachers  at  the  same  time  as  they  helped  them  develop  their 
personal  philosophies  of  teaching.  As  Sam  explained, 

I can  easily  say  that  my  exposure  to  David  helped  me  to  lay  the  foundation  of  my  educational  philosophy.  He 
gave  me  a valuable  reference  point  and  standard  to  aspire  to  ... . It  really  helped  me  throughout  the  semester  to 
have  David  in  the  back  of  my  mind  as  a baseline. 

A comment  in  Matt’s  journal  reiterates  Sam's  point:  "I  used  the  STF  system  as  a frame  of  reference  for  each  of  my  field 
experiences  in  one  way  or  another  throughout  the  entire  semester. 

In  addition  to  keeping  exemplars  of  good  teaching  in  the  backs  of  their  minds,  many  of  the  students  actually 
modeled  at  least  some  of  the  lessons  they  taught  in  their  field  experience  classrooms  after  lessons  they  had  viewed  on  the 
STF  videos.  For  example,  Sam  used  numerous  features  of  David  Birchfield’s  first  grade  measurement  lesson  in  designing 
his  own  first  grade  lesson  on  place  value.  Some  students  even  consciously  imitated  the  STF  teachers.  As  Matt  admitted, 
"I  guess  we  can't  get  much  closer  to  modeling  my  teaching  behavior  about  the  STF  than  this,  because  I basically  stole 

David  Birchfield’s  activity I basically  tried  to  be  David  Birchfield  in  this  lesson."  It  turned  out  that  Matt  found 

using  David  as  a mental  model  both  helpful  and  distracting,  as  comments  from  his  self-critique  interview  indicate.  Matt 
opened  the  discussion  of  his  teaching  video  with  the  following  self-analysis, 

I think  I was  able  to  be  relaxed  somewhat  in  front  of  the  class  as  a whole  in  this  video,  but  when  I was  working 
with  the  students  one-on-one,  I was  too  concerned  about  moving  around  the  room  and  trying  to  be  like  the 
model  STF  teacher. 

Though  Matt's  use  of  STF  as  a model  was  purposeful,  other  students  claimed  that  their  use  of  STF  models  was 
not  entireb  a conscious  decision.  For  example,  Tina  wrote  in  a journal  entry  that  she  was  surprised  how  much  STF 
images  affected  her  teaching: 

I \ 'as  very  critical  of  Vickie  at  the  beginning  of  the  semester.  But  only  a few  weeks  later  I gave  several  lessons 
that  were  very  much  like  hers  and  I returned  to  her  in  my  mind  as  a consultant  on  how  to  improve. 


Later,  in  the  interview  during  which  she  analyzed  her  own  teaching  video,  1 ina  elaborated  on  this  comment. 
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[At  first]  I hated  Vickie  Bill.  I just  hated  it.  I hated  it,  hated  it,  hated  it  — the  whole  approach.  ...  I still  have  a 
few  mixed  feelings  about  her  teaching,  but  the  thing  that's  funny  about  it  is  that  I feel  like,  in  a lot  of  ways,  my 
teaching  has  turned  out  a lot  like  hers. ...  As  I look  back.  I'm  seeing  more  and  more  the  merit  of  her  approach. 

Tina  was  not  the  only  student  whose  standards  changed  during  the  course  of  the  semester.  For  example,  Pete 
originally  thought  that  David  Birchfield  was  too  stem  with  his  students.  After  several  weeks  of  field  experience,  Pete 

changed  his  mind.  "When  I look  back  at  my  observations  of  David  Birchfield  I think  I was  way  too  severe I find 

myself  being  just  as  stern  as  he  was  in  the  video."  Matt  originally  thought  the  self-critiques  provided  by  STF  teachers 
were  too  analytic.  In  time,  he  came  to  value  the  attention  to  detail  that  these  critiques  provided. 

Early  ol  in  the  semester, ...  it  seemed  to  me  that  they  [STF  model  teachers]  were  too  critical  of  their  own 
teaching  style  and  reading  too  much  into  their  own  actions.  My  opinion  now  contrasts  shaiply  ...  It  seems  to 
me  that  attention  to  detail  is  a key  factor  in  understanding  the  effectiveness  of  one's  own  teaching 
skills/weaknesses. 

* After  teaching  in  his  field  experience  classroom  for  weeks  and  after  viewing  himself  on  videotape  Sam  really 
appreciated  the  exemplary  teaching  modeled  by  the  STF  teachers.  As  he  explained, 

I found  [STF  and  self-videotaping]  really,  really  useful. . . . They  [STF  teachers]  make  it  look  so  damned  easy  . 

. . then  when  I saw  my  videotape  and  compared  it,  it  was  just  like,  "my  god!"  There  was  such  a big  gulf  there.  . 

. . That  was  really  helpful.  I really  liked  being  able  to  compare  [my]  video  . . . and  the  other  ones. 

Donna  summed  up  the  students'  general  reactions  to  the  availability,  through  STF,  not  only  of  models  of  good 
teaching,  but  also  of  models  of  self  critique: 

I would  say  it  [STF]  is  most  useful  as  an  instructional  tool,  because  it  allows  you  to  see  other  teachers 
"perform"  in  the  classroom.  It  is  very  helpful  to  see,  and  hear,  how  teachers  develop  their  lessons  and  also  hear 
their  comments  on  classroom  management,  how  the  lessons  are  progressing,  etc. 

The  instructor  recalled  a specific  instance  where  David  Birchfield  served  as  a mental  model  during  a classroom 
discussion,  "When  we  dealt  with  the  session  on  counting  and  pre-number  ideas,  then  I know  oiat  I referred  to  the  things 
that  they  had  seen  in  David's  class  . . . [STF]  was  so  useful,  because  we  had  at  that  point  a common  reference  of  good 
teaching." 

Specifics  learned  from  STF.  The  STF  system  was  not  only  effective  in  providing  students  with  overall  models 
of  good  teaching,  but  also  in  providing  examples  of  a variety  of  specific  strategies  and  techniques.  For  example,  most  of 
the  students  had  very  little  experience  with  small  group  work  in  school,  but  they  were  anxious  to  try  it  in  their  teaching. 
They  appreciated  what  they  could  learn  about  this  new  pedagogical  technique  from  STF.  As  Matt  commented, 

[From  STF]  I was  able  to  learn  specific  groups  related  activities  that  I was  able  to  apply  to  math  activities  for 
my  second  grade  class.  ...  I [also]  drew  the  idea  of  assigning  different  roles  to  specific  group  members  from  the 
Dwight  Cooley  video  on  STF. 

The  students  also  mentioned  how  much  they  learned  about  effective  questioning  techniques  by  watching  the  STF 
videos  and  listening  to  the  related  perspectives.  Sam  put  it  this  way, 

I really  like  the  way  that  David  Birchfield,  when  he's  moving  from  group  to  group,  his  skill  at  questioning  them 
is  a lot  better  than  mine.  ...  I was  walking  around  and  kind  of  doing  the  same  trick  that  I learned  from  him, 
which  was  keeping  one  eye  open  on  everyone  else  and  watching  what  specific  ids  are  doing. ...  I really  identify 
with  the  kinds  of  things  I learned  about  from  David  Birchfield.  He  makes  that  kind  of  think  look  easy, . . . and 
it’s  not. 

Laura  appreciated  the  many  management  and  organizational  tips  she  gleaned  from  STF.  She  wrote,  "After 
watching  David  Biichfield,  I feel  even  more  strongly  about  organization. ...  He  had  certain  practices  to  identify 
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transitions,  like  a song  or  the  turning  off  of  the  light."  Matt  noted  that  he  really  had  never  given  much  thought  to 
classroom  management  before  observing  hearing  the  STF  teachers  explain  the  management  techniques  they  were  using: 

I feel  that  one  of  the  reasons  I was  able  just  to  manage  the  students  somewhat  effectively  in  this  lesson,  is 
because  of  watching  the  STF  video  system  and  just  getting  a better  understanding  of  how  teachers  can  move 
around  a classroom. ...  I was  never  really  aware  that  I could  actually  have  a strategy  of  moving  around  the 
classroom  before  watching  STF. 

He  also  noted  that  observing  via  STF  had  prepared  him  for  most  purposeful  and  effective  observing  in  his  field  experience 
classroom: 

Vickie  Bill  was  more  influential  in  terms  of  learning  effective  management  techniques  through  questioning, 
planning,  and  leadership.  After  watching  Vickie,  I was  able  to  notice  that  my  cooperating  teacher  had  a similar 
situation  of  effective  management  established  in  her  classroom.  I was  able  to  use  my  cooperating  teachers 
management  system  early  in  the  semester,  having  been  primed  for  such  a thing  by  watching  the  STF. 

■Stimulating  thinking  about  teaching.  Watching  STF  not  only  gave  the  preservice  teachers  specific  strategies  to 
use  in  teaching,  but  also  raised  a variety  of  issues  that  they  debated  in  class  and  wrote  about  in  their  journals.  For 
example,  many  were  anxious  to  know  how  Vickie  Bill  had  set  up  her  classroom  at  the  outset  of  the  school  year.  They 
admired  the  classroom's  organization  and  the  students'  clear  understanding  of  rules  and  expectations  (as  seen  in  the  video 
clips  of  Vickie's  teaching),  but  they  wondered  how  that  atmosphere  had  been  established.  As  Sam  put  it. 

The  most  intriguing  comment  I heard  on  the  disc  is  the  one  where  she  [Vickie]  states  that  the  first  5 days  are 
used  to  set  up  her  system  and  class  the  way  she  needs  to.  I want  to  know  exactly  what  she  tells  those  kids  and 
what  they  do  the  first  five  days  in  greater  detail.  I'm  sure  there  is  lots  of  valuable  info  there. 

Some  of  the  students  had  just  as  many  questions  for  David  as  for  Vickie.  The  students  seemed  to  think  of 
Vickie  and  Dav  as  experienced  colleagues  and  wished  they  were  available  for  personal  consultation  or  friendly  teacher- 
lounge  conversation.  Writing  in  his  journal.  Matt  expresses  their  sentiments  well:  "I  would  like  to  ask  David  a few 
questions.  I have  created  a list  that  I would  like  to  talk  to  him  about,  and  will  continue  to  add  to  it  for  the  rest  of  the 
semester." 

The  students  mentioned  numerous  ideas  obtained  from  STF  that  they  wanted  to  debate  or  to  learn  more  about. 

For  example,  they  were  generally  attracted  to  the  notion  of  creating  their  own  mathematics  curriculum  (rather  than 
relying  on  a textbook),  but  concerned  about  how  a teacher  can  actually  manage  to  do  this  in  practice  (and  whether  they , as 
novices,  could  pull  it  off).  In  one  of  the  STF  perspectives,  Vickie  Bill  explains  that  she  uses  student  logbooks  as  a 
record  of  the  ever  evolving  curriculum  in  her  classroom.  Gary  commented  in  his  journal, 

I love  [Vickie  Bill's]  idea  of  using  logbooks  for  texts.  I hope  after  I've  had  a few  years  of  experience  under  my 
belt,  I'll  have  enough  confidence  and  ambition  to  abandon  the  textbook  and  custom  design  my  math  lessons,  or 
at  least  many  of  them,  This  way  they  could  be  tailored  to  the  specific  needs  and  abilities  of  the  class. 

Laura  expressed  similar  sentiments: 

Another  problem  posed  by  STF  is  whether  or  not  to  even  use  a textbook  like  Vickie  did  not.  It  seemed  to  really 
work  for  her  and  her  students,  but  I know  at  this  time  I am  not  even  close  to  being  able  to  work  without  a text. 

Laura  also  noted,  with  a bit  of  surprise,  that  not  all  of  her  classmates  reacted  with  the  same  enthusiasm  to  Vickie  as  a 
model.  Laura  appreciated  a class  exercise  where  students  were  required  to  justify  their  opinions  about  the  effectiveness  of 
the  STF  teachers  by  pointing  to  specific  incidents  in  the  videos. 
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I thougnt  Vickie  Bill  was  more  effective  and  that  would  be  self  evident  to  everyone.  But  this  was  far  from  true. 
There  are  those  who  found  David’s  use  of  creativity  better  than  what  they  perceived  as  Vickie’s  overly  organized 
and  hard  to  follow  lesson.  By  making  them  and  me  give  specific  examples,  hopefully  we  are  better  able  to  see 
how  or  why  the  other  can  think  the  way  he  does. 

Summary  and  Discussion 

Our  focus  in  this  study  was  to  document  how  the  Strategic  Teaching  Framework  (STF)  could  be  used  in  a 
teacher  education  course  to  provide  models  of  teaching-in-action  about  which  prospective  teachers  could  reflect  and  debate. 
Our  goal  was  to  facilitate  the  development  of  teachers  comfortable  with  teaching  according  to  the  recommendations  of  the 
NCTM  Standards  and  thoughtful  about  their  development  as  flexible  practitioners.  Our  setting  for  the  study  was  the  field 
experience  component  of  a masters  certification  program  for  prospective  elementary  teachers.  Our  data  documented  how 
the  students  used  the  technology,  as  well  how  they  planned  and  taught  lessons  to  actual  classes  of  students  and  how  they 
reflected  on  teaching  and  learning  issues  in  class  discussions  and  in  their  individual  journals.  We  were  not  surprised  to 
observe  that,  as  novice  teachers,  the  PSTs  were  not  able  to  teach  lessons  that  came  up  to  the  standards  they  had  learned 
about.  Therefore,  documenting  the  PSTs'  growth  in  reflective  understanding  across  their  analyses  of  these  very  different 
types  of  teaching  episodes  was  of  particular  interest. 

We  are  still  interested  in  exploring  further  how  STF  could  best  be  utilized  in  this  course.  Its  use  at  the 
beginning  of  the  course  allowed  the  students  to  have  a "virtual"  field  experience.  Many  students  commented  on  its 
importance  in  providing  an  early  model  for  them  to  draw  upon.  However,  other  students  felt  that  the  time  spent  on  STF 
(four  hours  a week/three  weeks  in  row)  was  too  much.  We  wonder  if  STF  would  be  better  utilized  if  it  were  interspersed 
throughout  the  entire  class  as  opposed  to  loaded  at  the  front  end. 

One  of  the  students  (Laura)  devoted  much  of  her  journal  to  discussion  of  age  appropriateness.  In  particular,  she 
wondered  if  the  differences  she  saw  between  David  and  Vickie's  teaching  strategies  and  styles  were  due  to  the  differences  in 
the  ages  of  their  students.  Other  PSTs  also  mentioned  that  they  wish  they  had  a specific  exemplar  of  the  grade  level  they 
were  working  with.  We  are  curious  as  to  the  importance  of  specific  grade-level  exemplars.  How  close  to  the  students' 
actual  situation  do  video  models  need  to  be?  What  aspects  of  teaching  transfer  across  grade-levels  and  which  are  better 
served  by  specific  age  (or  for  that  matter,  content)  exemplars? 

In  any  case,  from  the  way  in  which  STF  was  incorporated  in  this  class  we  learned  the  following.  PSTs  were 
avid  users  of  the  STF  system,  prolific  and  thoughtful  writers  of  journal  entries,  and  eager  participants  in  discussions 
about  what  they  gained  from  use  of  STF.  In  a number  of  cases,  the  lessons  that  they  taught  in  the  schools  were 
explicitly  modeled  on  lessons  viewed  in  STF — and  in  their  self-critiques  of  these  lessons  the  PSTs  provided  rationale  for 
their  choice  of  lesson  and  reflected  thoughtfully  on  how  and  why  they  had  adapted  these  models.  Our  PSTs  reported  (and 
evidenced)  important  changes  over  the  course  of  the  semester  in  their  views  of  mathematics  and  of  what  constitutes 
effective  teaching.  The  teachers  in  the  STF  system  proved  to  be  important  models  for  the  PSTs,  while  the  commentary 
provided  within  the  system  provided  a springboard  for  student  reflection  and  discussion  both  in  their  journals  and  in  the 
class.  An  important  side  benefit  STF  provided  was  its  use  as  a common  image  and  reference  point  for  the  students. 
Throughout  the  semester,  students  were  able  to  refer  to  specific  instances  in  the  videodisks  or  comments  made  in  the 
commentaries  and  generate  discussion  widi  everyone  else  in  the  class  since  everyone  understood  the  reference. 

To  be  effective,  teacher  education  programs  and  classes  should  model  what  they  teach.  We  feel  that  integration 
of  STF  into  this  field  experience  provides  students  with  important  models  outside  of  the  teachers  on  the  videos.  First,  it 
models  an  appropriate  use  of  technology  that  allows  students  an  experience  they  could  not  have  had  otherwise.  The 
technology  in  this  instance  extends  their  boundaries  of  observation  and  experience  far  beyond  the  traditional  field 
experience,  allowing  them  to  see  and  communicate  (via  the  perspectives)  with  expert  teachers.  If  teacher  education 
classes  do  not  model  appropriate  uses  of  technology,  then  we  cannot  expect  new  teachers  to  develop  an  understanding  of 
technology's  potential,  for  both  good  and  bad, in  their  teaching.  Additionally,  the  constructivist  framework  underlying  the 
design  cf  the  software  is  consistent  with  the  philosophy  and  goals  of  the  Standards,  This  provides  another  important,  if 
subtle,  model  for  the  students.  As  they  work,  grow,  and  construct  their  own  ideas  about  teaching,  STF  provides  a 
learning  environment  which  exemplifies  the  goals  of  the  Standards.  Learning  in  this  environment  can  help  the  students 
understand  the  meaning  of  "constructing  knowledge"  and  help  them  to  be  better  able  to  create  similar  experiences  for  their 
students.  V/hile  we  recognize  that  it  may  be  impossible  to  isolate  the  influence  of  STF  from  other  influences  on  our 
PSTs  (methods  instruction,  class  discussion,  school  observations,  etc.),  we  feel  confident,  nevertheless,  that  use  of  STF 
contributed  significantly  lo  the  M501  field  experience. 
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Introduction 


Open-ended  learning  involves  self-directed  processes  that  are  driven  by  the  unique  intentions  and  purposes  of 
learners.  Contemporary  learning  theory  such  as  constructivism  (Jonassen,  1991),  situated  cognition  (Brown,  Collins,  & 
Duguid,  1989),  and  cognitive  flexibility  (Spiro,  Feltovich,  Jacobson,  & Coulson,  1991)  emphasize  the  active  role  of  the 
learner  in  constructing  understanding.  Furthermore,  theoretical  perspectives  about  student-centered  learning  have  become 
increasingly  foundational  for  contemporary  technology-based  environments  such  as  microworlds  (Papert,  1993)  and 
anchored  instruction  (Cognition  and  Technology  Group  at  Vanderbilt,  1992).  Open-ended  learning  environments  (OELEs) 
have  been  touted  as  one  approach  for  blending  learning  theory  and  emerging  technology  to  support  the  process  of 
building  and  evolving  student-centered  understanding. 

OELEs  use  technology  to  create  experiences  for  learners  to  build  and  test  their  intuitive  notions  about  the  world. 
Microworlds,  for  instance,  use  technological  environments  wherein  learners  can  begin  exploring  and  building  upon 
intuitive  models  (Rieber,  1992).  OELEs  are  based  on  an  assumption  that  learning  is  a continuous  and  dynamic  process 
that  constantly  evolves  as  a result  of  a learner's  observation,  reflection,  and  experimentation  (Hannafin,  Hall,  Land  & 

Hill,  1994).  OELEs  are  designed  to  provide  experiences  for  learners  to  identify,  question,  and  test  the  limits  of  their 
intuitive  beliefs  and  theories.  As  such,  the  learning  process  involves  developing  a theory -in-action-m  intuitive  theory 
that  is  generated  and  changed  by  learners  as  they  reflect  upon  experiences  that  either  confirm  or  challenge  the  validity  of 
the  theory  (Karmiloff-Smith  & Inhelder,  1975). 

Piaget  (1976)  characterized  the  learning  process  as  one  that  occurs  through  continuous  testing  and  reconciling  of 
beliefs  in  the  face  of  ever-changing  experience.  Understanding  is  constructed  from  initial  intuitions  that  are  built  upon  by 
learners  as  they  encounter  experiences  that  lead  them  to  question  the  adequacy  of  their  theory.  Once  the  theory  is 
questioned,  understanding  develops  as  learners  confront  their  theory  through  a series  of  actions  designed  to  test  it. 
Corrections  to  the  theory  can  be  made  as  learners  connect  thinking  and  action  by  testing  the  limits  of  the  theory. 

The  Problem 

The  process  by  which  learners  use  OELEs  to  build  and  evolve  new  models  of  understanding  has  not  been  studied 
extensively.  Despite  the  theoretical  ideals  of  OELE's,  the  process  by  which  learners  interact  with  these  systems  to  build 
an  initial  theory,  test  it,  and  revise  it  remains  unclear.  Some  studies  have  indicated  that  learners  do  not  use  system 
features  in  ways  conducive  to  building  understanding  (Atkins  & Blissett,  1992).  A greater  understanding  of  how  learners 
use  OELEs  to  build  and  evolve  theories-in-action  is  needed. 

Previous  research  has  provided  little  insight  into  the  process  used  by  learners  to  build  and  evolve  theories  or 
models  of  understanding.  This  study  was  less  about  examining  the  extent  of  a learner’s  understanding  than  about 
empirically  documenting  the  dynamic  process  of  conceptual  development  as  it  occurs  in  conjunction  with  an  open-ended 
learning  environment.  The  study  qualitatively  examined  how  the  organization  of  learner  actions  within  an  open-ended 
system  gave  rise  to  new  understanding  that  regulated  the  development  of  a theory-in-action. 

The  Purpose 

The  purpose,  of  the  study  was  to  investigate  how  theories-in-action  develop  in  collaboration  with  open-ended 
learning  environments.  The  study  examined  the  following  questions: 

1.  What  processes  are  used  by  learners  to  build  and/or  evolve  a theory-in-action? 

2.  What  intentions  are  used  by  learners  to  build  and/or  evolve  a theory-in-action? 

3.  How  do  learners  use  system  features  to  build  and/or  evolve  a theory-in-action? 

Method 

The  participants  were  4 seventh-grade  students  drawn  from  a general  science  class.  The  students  were  studied  as 
separate  cases.  The  learners  included  two  boys  and  two  girls.  The  OELE  was  the  ErgoMotion  level  III  interactive 
videodisk  (mechanical  physics),  which  combines  computer-generated  graphics,  computer  simulations,  video  and  print- 
based  materials.  The  content  of  the  OELE  is  physics  (force  and  motion),  and  learners  construct  understanding  of  these 
concepts  by  designing  a roller  coaster.  Learners  use  the  environments  by  manipulating  variables  such  as  amount  of 
mass,  curve  radius,  and  hill  sizes.  Informational  resources  are  also  embedded  into  the  program  such  as  on-line 


consultants  and  a "videopedia."  The  primary  techniques  for  collecting  data  included  think-aloud  protocols,  observations, 
and  interviews. 

Learners  used  the  system  for  approximately  four  hours,  and  the  sessions  were  videotaped  and  later  transcribed. 
Interviews  took  place  approximately  one  week  later,  and  learners  were  given  analog  problems  of  physics  concepts,  a 
perception  of  open-ended  learning  questionnaire,  and  a feature  knowledge  questionnaire.  Learners  were  led  through 
videotapes  of  their  sessions,  and  were  asked  to  clarify  their  thinking  and  intentions.  Interviews  lasted  approximately  2 
hours. 

Findings  and  Implications 

Findings  were  organized  according  to  an  analysis  of  system-based  events,  learner  processing  of  events,  and 
learner  intentions  for  action.  The  analysis  used  detailed  data  from  verbal  protocols  to  examine  individual  actions  and  the 
intentions  and  processes  that  gave  rise  to  them.  The  goal  of  the  analysis  was  to  identify  and  represent  how  actions, 
processes,  and  intentions  formed  on  a micro-level  during  open-ended  learning,  in  order  to  derive  a theoretical  framework  to 
better  understand  cognitive  processes  at  a macro-level.  The  data  were  assigned  to  categories  that  represent  the  extent  to 
which  a learner  builds  and  evolves  a theory-in-action. 

Question  1 : What  processes  are  used  bv  learners  to  build  and/or  evolve  a theory-in-action? 

This  question  examined  the  extent  to  which  a learner  processed  system  feedback  resulting  from  taking  action. 
Three  levels  of  processing  distinguished  the  ways  that  learners  processed  information  during  theory  development. 
Specifically,  learners  perceive  relevant  conceptual  information,  organize  the  information  around  interpretations  or 
explanations,  and  integrate  information  with  existing  prior  knowledge  (Mayer,  1989).  Responses  were  identified  as 
instances  of  processing  if  they  represented  cognitive  reactions  to  information  or  feedback  provided  by  the  system. 

Perception.  At  a minimum,  all  learners  processed  information  at  the  perceptive  level.  Most  learners  recognized 
the  effects  of  their  actions  on  system  events.  For  instance,  one  learner  perceived  visually  that  variations  in  mass  and 
horsepower  resulted  in  less  time  for  the  coaster  to  reach  the  top  of  the  first  hill: 

I thought  that  the  more  mass  that  was  in  there  [with  a 50  versus  a 25  horsepower  engine],  it  took  it  a while  to 

get  up  there.  So  I'm  now  going  to  change  the  horsepower  to  100. 

In  this  case,  he  selected  the  information  that  was  relevant  (i.e.,  changed  the  horsepower),  and  derived  conclusions 
regarding  its  effect  on  the  coaster.  This  learner  used  observational  data  generated  by  the  system  to  draw  a valid  conclusion 
about  this  observation. 

Learners  often  made  valid  conclusions  concerning  their  observations,  in  the  form  of  either  success  or  failure  of 
the  coaster,  or  verbal  or  numerical  information  provided  by  the  system.  On  the  other  hand,  learners  frequently  drew 
erroneous  conclusions  based  upon  their  observations.  Learners  frequently  confounded  visual  information  with  judgments 
based  upon  subjective,  naive,  incomplete  or  inaccurate  visual  information.  Confounded  observations  were  frequently  used 
to  describe  the  behavior  of  the  coaster  in  subjective  and  inaccurate  ways  (e.g.,  "It’s  gaining  speed  at  the  very  end").  For 
instance,  after  lowering  the  horsepower  and  observing  the  coaster  crash,  one  learner  concluded,  "It  went  further  this  time 
[around  the  curve]."  Objective  data  for  drawing  valid  conclusions  (i.e.,  the  coaster  crashed)  were  confounded  with 
subjective  perceptions  of  visual  information  (i.e.,  it  went  further).  In  this  instance,  perceptions  of  inaccurate  data  resulted 
in  erroneous  assumptions  about  the  effect  of  horsepower  on  the  speed  of  the  coaster  (i.e.,  it  went  further  after  lowering 
the  horsepower;  therefore,  horsepower  must  affect  speed). 

Organization.  Most  learners  attempted  to  assign  meaning  to  system  events  by  generating  interpretations, 
expectations,  or  evaluations  based  on  system  feedback.  Learners  used  organizational  processes  to  establish  simple  cause- 
effect  relationships  and  expectancies.  For  instance,  they  learned  quickly  that  when  the  coaster  moved  rapidly  during  the 
simulation,  it  was  likely  to  crash.  Thus  they  formed  a simple  expectation  that  was  easily  confirmed,  based  on  whether  or 
not  the  coaster  crashed.  Statements  of  expectancy  such  as  "It's  going  to  fall  off,"  "I  didn’t  think  it  would  make  it,"  and 
"I  knew  it  would  go  over"  were  used  often  used  to  establish  and  confirm  rudimentary  expectations  about  whether  or  not 
the  coaster  would  "work." 

Eventually,  learners  used  information  from  the  system  to  form  simple  cause-effect  relationships.  They  provided 
a reason  or  cause  for  a system  event  (e.g.,  "Yeah,  it  was  probably  too  much  horsepower  going  up...").  Simple  cause- 
effect  interpretations  were  usually  based  on  intuitive  ideas  about  the  cause  of  a coaster  crash.  For  instance,  one  learner 
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tried  to  stop  the  coaster  from  crashing  for  several  trials  before  offering  an  explanation:  ”1  figured  out  what  I did  wrong. 
There  wasn't  enough  weight.”  Based  on  this  interpretation,  she  took  action  to  increase  the  mass.  In  a similar  instance, 
another  learner  remarked,  "I  think  the  engine's  too  [powerful],”  and  lowered  the  horsepower  accordingly.  In  these 
instances,  learners  offered  an  explanation  for  the  event,  but  did  not  provide  observational  evidence  to  support  their 
interpretation.  However,  the  interpretation  or  expectation  was  used  to  take  future  actions. 

Learners  frequently  interpreted  system  feedback  based  upon  preconceived  assumptions  or  beliefs.  To  illustrate, 
one  learner’s  subjective  interpretation  of  system  data  was  evident  in  his  statements  about  the  role  of  mass  and  speed:  "I 
think  that  with  more  people,  it  went  a little  bit  slower  than  what  it  did  without  less  people  ...  The  more  mass  you  have 
on  it,  like  the  slower  it  will  go.”  During  his  next  interaction,  he  changed  the  hill  sizes  ”to  get  more  speed”  and  decreased 
the  mass.  He  remarked. 

I'm  going  to  try  it  with  no  people  on  it. ..with  the  same  engine  ...[runs  simulation;  coaster  is  successful] 

Hmm...  It  went  faster  than  the  one  with  the  people  on  it.. ..Yeah,  I think  it  might  have  [gone]  faster  than  the 
one  with  people  on  it,  but  because  of  the  hill  size  I have,  with  the  smaller  hill  up  here  (hill  3),  it  went  slower,  a 
little  bit,  than  it  could  have  went. 

In  this  instance,  he  appeared  to  interpret  visual  cues  from  the  system  in  ways  that  were  consistent  with  his  intuitive 
assumptions  about  factors  affecting  the  speed  on  the  coaster  (in  this  case,  mars).  The  consequence  of  his  intuitive 
perceptions  was  faulty  interpretations  regarding  causes  and  effects.  In  such  situations,  it  appeared  that  learners  "saw” 
what  they  intuitively  expected  to  take  place,  and  consequently  derived  perceptions  and  interpretations  of  visual  cues  that 
were  consistent  with  these  assumptions.  Consequently,  learners  were  often  unable  to  use  system  feedback  to  further 
revise  and  refine  interpretations. 

Learners  also  elaborated  and/or  confirmed  expectations  using  intuitive  assumptions  and  beliefs  not  generated 
from  objective  data.  They  tended  to  elaborate  and  change  interpretations  in  ways  that  were  consistent  with  their  intuitive 
expectation.  When  an  expectation  was  not  confirmed  by  system  feedback,  learners  often  changed  or  added  to 
interpretations,  in  order  to  make  them  consistent  with  the  new  data.  For  instance,  one  learner  encountered  the  following 
actions,  each  of  which  was  sparked  by  inconsistencies  between  observed  data  and  expectancies: 

I'm  going  to  put  more  weight  on  there.  So  it’s  like  a. full  cart,  I guess,  [runs  simulation]  It’s  going  to  go  off 
that  track,  [crashes].  I guess  you  just  can’t  have  that  much  power  on.  I'm  going  to  put  a smaller  engine  on 
there...  So  with  a smaller  engine,  I think  it  will  stay  on  the  track,  [crashes].  I think  since  it  had  so  many  people 
on  it,  ...  it  makes  a lot  of  force  ...  with  all  the  weight....  I'm  going  to  try  it  with  less  people...  Maybe  it  will 
stay  on.  [crashes]  ....  I guess  ..  even  with  the  smallest  engine,  all  that  weight  on  it,  is...  playing  a big  part  in 
making  it  go  off....  With  an  engine  with  no  people  on  it,  I don't  think  it  would  go  off.  Yeah,  if  it  didn't  have 
any  people  in  the  cart,  it  wouldn't  go  off.  I'm  going  to  try  it  with  a little  less  people  in  it.  [crashes]  Now  I 
know  ...  now  I think  it's  like  ...  I think  it's  the  hills  [italics  added]  now.  Cause  even  with  less  people  on  there 
...  with  the  real  steep  hills,  it's  getting  too  much  speed. 

When  expectations  were  not  confirmed,  this  learner  appeared  to  either  create  new  expectations  or  add  to  the  previous  ones 
in  ways  that  preserved  his  underlying  assumptions.  As  inconsistent  data  continued  to  challenge  his  assumptions,  his 
expectations  about  the  importance  of  mass  and  horsepower  may  have  weakened,  but  were  not  altered.  This  was  further 
evidenced  by  an  interaction  that  took  place  shortly  after,  when  he  tried  to  determine  why  the  coaster  did  not  ascend  the 
third  hill. 

It  didn’t  even  have  enough  power  to  make  it  up  that  real  steep  hill.  So,  I'll  make  [hill  1]  steeper... [goes  to 
energy  loading]  Well  let  me  try  something,  [changes  horsepower  from  25  to  50]  I think  it  might  have  more 
power  now,  and  then  I can  leave  it  like  that  and  it  might  go  over,  [coaster  does  not  make  it  over  hill  3]...  Well, 
I think  I'll  not  make  [hill  1]  the  steepest  ...  but  make  it  a little  bit  steeper. 

He  returned  to  his  assumptions  about  horsepower  to  interpret  events  and  guide  future  actions.  These  examples  suggest 
that  powerful  intuitive  assumptions  drive  both  interpretations  about  the  reasons  for  actions  and  decisions  about  future 
actions,  and  are  not  easily  altered. 
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Integration.  Integration  involved  connecting  actions,  concepts,  or  system  events  to  prior  knowledge  or  relevant 
personal  experiences,  which  were  used  to  interpret,  evaluate,  or  supplement  conceptual  understanding,  Integration 
relationships  were  evident  in  references  to  analogies,  metaphors,  or  concrete  personal  experiences  (Mayer,  1989).  In 
comparison  to  other  processing  levels,  learners  showed  little  evidence  of  integrating  system  events.  Those  who  integrated 
often  used  their  experiences  of  riding  roller  coasters  to  elaborate  the  data.  Sometimes,  however,  prior  experiences  with 
roller  coasters  contradicted  the  system  experiences.  For  instance,  the  solution  to  one  of  the  "coaster  challenge"  problems 
was  to  arrange  the  sizes  of  three  hills  so  that  the  coaster  would  roll  backwards  down  hill  3,  roll  backwards  over  hill  2,  and 
come  to  a rest  in  valley  1 , One  learner  initially  wanted  to  attempt  this,  but  remarked: 

Oh  yeah.  It’s  got  brakes,  it  cant  roll  backwards  ...  They  told  us  that  because  there  was  this  one  lady  who  was 
freaking  out  before  we  went  on  a ride.  And  they  said,  "it's  got  automatic  brakes  along  the  edges,  and  it  if  stops, 
it  will  clamp  and  you'll  hang." 

She  continued  to  make  references  to  brakes  and  clamps  during  remaining  use  of  the  system  when  addressing  issues  of 
slowing  down  and  stopping.  Prior  knowledge,  in  this  case,  appeared  to  limit  interpretations  and  future  actions  due  to 
inconsistencies  between  personal  experience  and  the  environment. 

Two  learners  related  observations  or  interpretations  to  personal  experiences.  In  reference  to  information  provided 
by  a consultant  on  acceleration,  one  learner  remarked:  "I  was  thinking  about  when  you're  in  a car  or  something,  how 
when  you  like  go  faster  ...  it  will  like  push  you  back  in  your  seat,  and  when  you  come  to  a stop  sign,  you  will  go 
forward."  After  listening  to  information  from  the  on-line  consultants  on  "what  affects  acceleration,"  he  added:  "I've  got  a 
dirt  bike,  and  when  you  go  faster,  it  kinda  pushes  you  back  ...  like  pushing  on  you  a little  bit."  In  this  instance,  he 
clearly  linked  the  concepts  presented  in  the  system  with  his  own  personal  experiences. 

Question  2:  What  are  a learner's  reasons  intentions  for  building/evolving  a theorv-in-action? 

Intentions  represent  a reason  for  action.  Intentions  can  mediate  a learner's  actions  and  the  processing  of 
information  resulting  from  them.  Three  levels  of  intention  distinguished  the  ways  that  learners  used  information  to 
regulate  future  actions:  Unsystematic,  goal-based,  and  means-based.  In  unsystematic  searches,  learners  browsed  the 
system  with  no  apparent  intention  to  either  meet  a specific  goal  or  understand  relationships.  With  goal-based  intentions, 
learners  focused  on  taking  specific  action  to  meet  a specific  goal  (i.e.,  to  make  the  coaster  "work");  Means-based 
approaches  focused  on  taking  action  to  discover  why  an  event  is  occurring  or  what  could  happen  if  limits  were  extended. 
The  search  for  means  implies  a goal  of  confirming  or  refuting  a theory  (Karmiloff-Smith  & Inhelder,  1975). 

Unsystematic  searches.  All  learners  conducted  unsystematic  searches  during  the  learning  process.  Such 
intentions  often  took  the  form  of  "browsing"  the  system  features  and  trying  to  determine  their  functions  (e.g.,  "...see 
what  this  does.”).  Often,  unsystematic  intentions  were  manifested  in  situations  where  uncertainty  existed  about  how  to 
solve  a problem.  For  instance,  one  learner  explored  the  menu  bar  and  attempted  to  use  a video  grabber  tool  to  help  him 
solve  the  problem.  He  stated,  "I  don't  know  what  that  is  ...  I'll  try  it  I guess."  Still  unable  to  solve  the  problem,  he  later 
tried  the  grabber  tool  again,  and  frustrated,  remarked,  "What  is  that  grabber  [italics  added]?" 

Goal-based  intentions.  The  focus  of  goal-based  intentions  was  to  find  out  what  "worked"  and  to  take  subsequent 
actions.  Learners  often  set  goals  and  took  actions  to  meet  them,  noting  actions  that  did,  and  did  not,  prove  successful. 

For  instance,  after  a coaster  crash,  one  learner  remarked,  "Well,  I found  out  what  hills  don't  work  here."  He  then  wrote 
down  his  "answer"  and  left  the  coaster  site  shortly  thereafter  to  "[go]  back  to  my  questions  and  answer  another  one."  In 
these  instances,  goal-based  approaches  were  used  test  ideas  about  what  might  work.  The  reasons  for  success  were  not 
addressed;  rather,  information  about  the  success  was  noted  and  recorded  for  later  reference. 

Means-based  intentions.  All  learners  demonstrated  means-based  intentions,  but  to  varying  degrees.  Means-based 
intentions  were  identified  when  learners  developed  new  intentions  to  explore  the  limits  and  boundaries  of  actions,  rather 
than  focusing  on  what  is  "known"  to  solve  the  problem.  For  instance,  one  learner  had  the  following  interaction: 

[coaster  crash]  I figured  that  might  happen.  Yeah,  it  was  probably  too  much  horsepower  going  up  and  all ... 
When  you  first  start  off  going  up  the  hill,  it's  pushed  so  fast,  and  you’ve  got  all  that  force  from  the  first  hill, 
because  I made  it  real  steep  ...  [changes  horsepower  to  25]  I made  it  so  I dropped  the  horsepower  of  the  engine, 
and  I made  the  first  hill  smaller,  so  it  will  probably  stay  on  the  track,  [coaster  makes  it] 
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In  this  instance,  Jason  elaborated  a clear  expectation  or  theory  as  to  why  the  coaster  crashed,  and  offered  an  alternative  he 
anticipated  to  result  in  success.  Following  success,  he  continued  exploring  alternative  solutions. 

[makes  curve  small  and  runs  simulation]  I think  it's  going  to  go  off  the  track,  [crashes].  Well,  yeah,  with  that 
sharp  of  a curve,  and  it  had  that  much  speed  going...  I'm  going  to  make  [the]  hills  smaller,  so  maybe  even  with 
the  sharper  curve,  it  will  stay  on  the  tracks,  [coaster  makes  it]  I kinda  Figured  that  one  [would  work],  'cause 
...the  first  hill  was  not ...  so  steep...  [changes  mass  to  6000],  I'm  pretty  sure  this  one  will  stay  on.  With  that 
many  people  on  there,  it  will...  make  it  slower  and  all.  Yeah,  and...  maybe  also  because  [the  first  hill]  wasn't 
very  steep. 

During  this  interaction,  he  acknowledged  that  the  sharpness  of  the  curve  caused  the  coaster  crash.  However,  rather  than 
changing  the  curve,  which  would  be  the  likely  choice  for  success,  he  changed  the  hill  sizes  to  explore  the  limits  of  the 
curve's  capabilities.  In  this  instance,  this  learner  appeared  to  evolve  his  intentions  from  choosing  the  most  likely 
solution  to  finding  alternative  solutions. 

Question  3:  How  do  learners  use  system  features  to  build  and/or  evolve  a tlieory -ip-action? 

Feature  use  distinguished  the  ways  learners  responded  using  the  features  and  functions  of  the  system.  Feature  use 
was  defined  operationally  as  the  way  in  which  learners  used  the  system  to  meet  a goal  or  intention.  Three  categories  were 
developed:  (1)  general  awareness  of  the  existence  and  function  of  a feature;  (2)  awareness  of  how  to  use  a feature  to 
achieve  a desired  goal;  (3)  awareness  of  how  to  use  a feature  to  build,  test,  or  evolve  a theory. 

General  awareness  of  existence  and  functions  of  a feature.  During  the  interviews,  learners  were  asked  questions 
about  their  awareness  of  system  features.  All  indicated  at  least  minimal  awareness  of  primary  system  features.  Even 
when  system  features  were  not  used,  learners  responded  in  the  interviews  that  they  knew  that  they  existed.  For  instance, 
none  of  the  learners  except  one  used  the  coaster's  features  to  gather  numerical  information  about  the  speed,  g-forces, 
kinetic  and  potential  energy,  and  acceleration  at  given  points  along  the  track.  One  learner  explained  his  awareness  and 
reasons  for  not  using  them: 

Researcher:  There  are  data  points  under  the  options  menu,  and  it  puts  these  little  hot  spots  all  along  the  .. 
Learner:  Oh  yeah,  yeah. 

Researcher:  So  you  knew  about  that.  But  you  didn't  use  it. 

Learner:  Yeah  ...  I guess  I just  didn't  think  I needed  those  data  points  to  help  me.  I think  that  if  I really  thought 
about  it,  I could  just  figure  it  out  on  my  own. 


Another  learner  responded  in  a similar  manner,  noting  " ..  I guess  I just  thought  I could  manage  without  them."  Such 
responses  may  imply  that  learners  view  system  features  other  than  primary  manipulation  tools  as  needed  only  for 
assistance. 

Awareness  of  how  to  use,  system  features  to  meet  a goal.  Feature  usage  was  most  apparent  in  situations  where 
learners  verbalized  a specific  intentor  goal,  and  then  told  how  they  would  use  the  system  to  achieve  it.  At  other  times, 
learners  perceived  that  the  system  could  not  provide  the  information  needed  to  meet  a goal.  This  was  evident  even  when 
the  system  could  provide  the  information  desired,  but  learners  were  unable  to  recognize  that  the  information  was 
accessible.  For  instance,  one  learner  stated,  "I  wish  I knew  what  motor  we  were  using  on  this."  [Although  the 
information  was  not  displayed  at  that  moment,  it  was  available  in  a different  section  (at  the  energy  loading  section)]. 

This  learner  did  not  know  how  to  maneuver  the  system  resources,  tools,  and  features  in  order  to  solve  her  problem,  even 
though  she  was  aware  that  the  features  existed.  In  this  case,  the  system  provided  neither  the  information  she  sought  nor 
cueing  to  possible  features  for  answering  her  questions. 

One  learner  encountered  a situation  where  the  information  was  immediately  available,  but  was  not  represented  in 
a form  she  could  recognize.  As  the  following  interaction  illustrates,  she  did  not  find  the  information  useful: 

[sets  hill  heights  at  hills  and  valleys  section]  Yeah,  that  would  be  Fine,  it  would  just  take  them  longer  ...  to 
cover  the  track.  I wish  ..  it  would  tell  me  how  much  time  it  would  take  ...  Well,  1 can  look  at  the  data  points, 
[views  data  points,  then  runs  simulation].  Oh  I wish  it  would  tell  me  the  time  it  took  for  it  to  run  the  lull 
course  ...  Then  I'd  be  able  to  tell  which  one  was  faster. 
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The  information  on  the  roller  coaster's  speed  was  available  for  each  point  along  the  track.  This  learner,  however,  sought  a 
specific  form  of  the  data,  one  that  was  not  represented. 

Awareness  of  how  to  use  system  features  to  build,  test,  or  evolve  theory.  Awareness  of  how  to  use  the  system 
to  develop  and  test  hypothetical  situations  was  not  apparent.  Learners  offered  theories  or  reasons  for  the  consequences  of 
an  event,  held  that  predicted  variable  constant,  and  intentionally  tested  it  using  manipulation  tools  of  the  system. 
However,  they  did  not  use  system  features  to  derive  and  test  hypothetical  problems  or  to  derive  counter-examples  to 
confirm  or  refute  a theory. 

One  learner,  for  instance,  often  used  the  system  to  test  and  revise  hypotheses: 

[sets  hills]  I'm  not  sure  if  it  will  even  make  it  over  the  hill,  [coaster  stops  at  top  of  hill  3]  ...  'cause  that  first 
hill  is  like  shorter  than  the  third  hill  ...[changes  hill  1 to  medium;  stops  at  top  of  hill  3]  Still  didn't  make  it. 
Add  more  horsepower  to  it  [changes  horsepower  from  25  to  50].  Still  [italics  added]  didn't  make  it ...  Change  the 
weight  on  it  [changes  mass  from  6000  to  4000  kg].  I think  it  will  make  it  over  now.  [stops  at  top  of  hill  3]. 
Yeeee...  nope,  [changes  mass  to  20000  kg.;  still  does  not  go  over  hill  3]...  Well,  if  I change  the  horsepower,  it 
might  make  it  over  the  hill,  [changes  horsepower  to  100].  [stops  at  top  of  hill  3].  Still  didn’t  do  it.  Change  the 
...  hills  I guess. 

After  lowering  hill  3,  the  coaster  run  was  successful.  However,  despite  evidence  that  could  be  used  to  question  the 
validity  of  his  horsepower  theory,  this  learner  did  not  use  the  system  to  confront  his  faulty  assumptions.  That  is,  he  did 
not  try  to  construct  a hypothetical  counter-example  to  see  if  the  horsepower  affected  acceleration  in  the  ways  he  believed. 
It  is  also  interesting  to  note  that  this  learner  successfully  answered  a question  in  the  Science  Talk  Show  about  the 
whether  doubling  the  size  of  the  motor  was  necessary  if  doubling  the  hill  size.  It  appears  that  he  acknowledged  the 
relationship  between  horsepower  and  acceleration  on  an  abstract,  conceptual  level,  but  did  not  recognize  the  information 
as  counter  to  his  theories  about  horsepower.  Due  to  apparent  difficulty  in  recognizing  counter-examples,  it  is  not 
surprising  that  learners  failed  to  use  the  system  to  systematically  construct  and  test  them. 

General  Discussion 

The  process  of  developing  theories-in-action  in  conjunction  with  OELEs  appears  to  center  around  three  primary 
areas:  Decisions  about  how  to  use  the  system,  processing  of  system-generated  feedback,  and  intentions  for  further  action. 
The  more  closely  linked  these  three  components,  the  more  likely  a theory-in-action  will  develop  and  evolve.  The  model 
presented  in  Figure  1 illustrates  the  interplay  between  the  system  and  the  learner  in  developing  a theory-in-action. 
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Figure  Captions 


Figure  1 : A conceptual  model  of  the  theory-in-action  development  process 


The  model  predicts  that  as  learners  progress  in  their  understanding,  underlying  theories  are  built,  confirmed, 
and/or  refuted.  An  initial  theory-in-action  is  built  iteratively  through  basic  levels  of  processing,  perception  and 
interpretation,  which  are  informed  through  learner  intention  and  action.  Learners  perceive  relevant  information  from  the 
system,  use  their  intuition  and  prior  knowledge  to  provide  interpretations  to  explain  system  events,  and  establish 
intentions  to  take  action  using  system  features  to  test  their  interpretations.  A refined  theory-in-action  evolves  as 
learners:  (1 ) perceive  information  as  consistent  or  conflicting  with  their  theory;  (2)  interpret  a new  theory  or  elaborate  an 
existing  one;  (3)  evaluate  the  validity  of  the  theory  using  new  data;  (4)  extrapolate  alternative  rules  or  explanations;  (5) 
use  intentions  to  expand  the  predictive  boundaries  of  the  theory;  and  (6)  take  action  to  test  their  previously-held  or  new 
theory.  The  study  uncovered  several  prominent  findings  with  significant  implications  for  understanding  how  learners 
build  and  evolve  theories-in-action  with  OELEs. 

The  Resilience  of  Theories-in- Action 

An  important  implication  of  the  study  is  that  theories-in-actions  were  extremely  resistant  to  change.  Whereas 
learners  showed  evidence  of  building  theories-in-action,  they  did  not  appear  to  evolve  them  significantly.  Through 
interaction  with  the  system,  learners  became  more  aware  of  their  intuitive  theories  by  using  them  to  explain  system 


O 

ERIC 


t v. 


392 


405 


events;  they  did  not,  however,  recognize  limitations  in  their  predictive  value  and  attempt  to  refine  them  further.  Learners 
hold  powerful  personal  theories  that  often  dominate  cognitive  processes  and  actions  with  OELEs.  Even  with  system 
tools  to  manipulate  and  evolve  conceptions,  initial  learner  theories  remained  largely  unchanged.  Learners  did  not  use 
system  tools  to  manipulate  the  theories;  instead,  they  tended  to  manipulate  the  system  and  interpretation  in  order  to 
preserve  them. 

The  Over-Reliance  of  Visual  Cues  from  the  System 

Another  prominent  finding  was  that  learners  relied  heavily  on  the  video  simulations  to  provide  feedback  about  a 
previous  action.  Learners  used  the  video  segments  literally  to  estimate  the  speed  of  the  coaster  during  the  simulation  and 
the  probability  of  its  crashing  (if  it  was  moving  rapidly  on  the  simulation,  it  would  crash).  Learners  also  used  visual 
cues  to  interpret  the  speed  of  the  coaster,  and  judge  relative  differences  in  speed. 

A reliance  on  the  literal  interpretation  of  visual  cues  is  likely  due  to  novice  learners1  focus  on  superficial  or 
surface  features  of  a problem  (Chi,  Glaser,  & Rees,  1982).  It  is  often  difficult  for  novices  to  select  relevant  information, 
and  they  typically  interpret  visual  symbols  as  relevant  (Petre,  1995).  Accordingly,  novices  confuse  visibility  with 
relevance,  and  focus  on  obvious  visual  cues,  without  considering  the  underlying  logic  of  their  selection.  This  tendency  is 
possibly  influenced  by  frequent  exposure  to  television,  videos,  and  computer  games,  where  learners  have  adapted  to 
processing  obvious  visual  information  without  considering  deeper  meaning. 

In  the  present  study,  reliance  on  visual  cues  was  damaging  to  theory  building  because  the  system  feedback 
provided  inaccurate  simulations  of  speed.  Consequently,  learners  misperceived  the  fidelity  of  the  images,  and  as  such, 
built  initial  assumptions  and  theories  that  were  incorrect  and  difficult  to  alter.  Without  accurate  and  more  precise 
representations  of  speed,  learners  could  not  evaluate  limitations  in  their  thinking.  When  ambiguously-presented  data  are 
interpreted  literally  by  learners,  theory  evolution  is  stunted  due  to  misrepresented  feedback  ultimately  perceived  by 
learners  as  valid. 

Another  consequence  of  relying  on  ambiguous  and  inaccurate  visual  cues  was  to  "contaminate"  or  bias  learner 
interpretations  (Wilson  & Brekke,  1994).  In  the  present  study,  learners  often  perceived  inaccurate  and  ambiguous  visual 
information,  and  used  it  to  preserve  or  confirm  their  thinking.  For  instance,  learners  typically  believed  that  increasing 
the  mass  increased  the  coaler's  acceleration.  After  running  the  simulation,  they  often  inaccurately  concluded  that  the 
video  feedback  supported  the  belief  (e.g.,  "...it  went  faster  than  the  one  with  less  mass").  Consequently,  over-reliance  on 
visual  cues  not  only  influenced  inaccurate  theory  building,  it  also  preserved  i*. 

The  Situated  Learning  Paradox 

One  assumption  underlying  the  design  of  OELEs  is  that  learning  is  optimized  when  it  is  situated  in  an  authentic 
context  (Hannafm  et  al.,  1994).  Learning  is  facilitated  when  experiences  to  connect  prior  and  everyday  knowledge  to  new 
contexts  are  provided  (Choi  & Hannafin,  1 l95).  As  learners  connect  system-derived  experiences  with  prior  knowledge, 
they  access  existing  frameworks  to  be  built  upon  and  used  to  support  new  understanding.  This  knowledge  can  then  be 
used  to  enrich  or  elaborate  system  experiences  and  intuitive  theories. 

OELEs  use  authentic  contexts  to  facilitate  the  linking  of  system  events  to  prior  knowledge.  For  instance,  the 
roller  coaster  site  is  authentic  in  that  it  embeds  relevant  knowledge  and  skills  and  assists  learners  in  connecting  system 
experiences  to  prior  knowledge  (most  children  are  familiar  with  roller  coasters).  While  links  to  prior  knowledge  enhance 
the  potential  for  transfer  (Brown,  et  al.,  1989),  it  also  increases  the  likelihood  that  learners  will  draw  upon  incomplete 
and  often  inaccurate  understanding  which  form  the  basis  of  faulty  theories.  Findings  from  this  study  indicated  that 
learners  often  referenced  prior  knowledge  and  experiences  that  either  contradicted,  or  interfered  with,  the  system’s 
treatment  of  the  concepts  of  force  and  motion. 

Fragmented  prior  knowledge  affects  how  information  is  interpreted  and  actions  are  taken.  For  instance,  in  the 
present  study  one  learner  recalled  a roller  coaster  operator  telling  her  that  the  coaster  had  brakes  and  could  stop  mid-ride. 
Consequently,  she  used  this  information  to  interpret  feedback  and  drive  future  actions.  As  a result,  she  often  interpreted 
the  coaster  as  slowing  down  around  curves  because  of  her  belief  that  the  coaster  used  brakes.  In  this  case,  a belief  that  the 
coaster  slowed  down  around  the  curve  (because  of  brakes)  interfered  with  an  understanding  of  force  and  motion  in  a 
context  that  did  not  support  exploration  of  the  notion. 

In  sum,  despite  evidence  of  evolution  of  theories-in-action,  this  study  indicates  that  newly  evolved  theories  are 
fragile  and  easily  abandoned.  The  strength  and  persistence  of  intuitive  theories-in-action,  in  the  face  of  conflicting 
experiences,  raises  questions  regarding  the  nature  and  process  of  learner  cognitive  development. 
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Introduction 


Traditional  instructional  approaches,  as  well  as  the  systems  design  processes  that  support  them,  have  been 
criticized  for  failing  to  reflect  contemporary  thinking  about  teaching,  learning,  and  technology.  Several  perspectives  have 
emerged  regarding  the  role  of  instructional  systems  design  in  supporting  critical  thinking  and  problem  solving.  Many 
believe  that  instructional  design  methodologies,  themselves,  are  not  inherently  limiting.  Limitations,  it  is  argued,  result 
from  narrow  interpretation  rather  than  inherent  shortcomings  in  the  approaches  themselves  (Reigeluth,  1989).  Others 
advocate  extending  or  adapting  traditional  design  methodologies  to  better  accommodate  diverse  perspective:  and 
contemporary  research  and  theory  (Lebow,  1993;  Rieber,  1992).  Still  others  disagree,  noting  that  many  objectivist 
assumptions  and  pedagogical  requirements  of  "instruction”  are  fundamentally  incompatible  with  non-objectivist 
viewpoints  (Cunningham,  1987;  Kember  & Murphy,  1990).  The  need  for  enlightened  uses  of  technology  to  support 
alternative  approaches  is  apparent.  Student-centered  learning  environments  have  been  touted  as  one  alternative. 

Student-centered  learning  environments  provide  interactive,  complimentary  activities  that  enable  individuals  to 
address  their  unique  learning  interests  and  needs,  examine  content  at  multiple  levels  of  complexity,  and  deepen 
understanding  (Hannafin,  1992).  Contemporary  environments  create  multidimensional,  ecologically  valid  systems  where 
students  access  existing,  or  build  new,  conceptual  linkages.  They  establish  conditions  that  enrich  thinking  and  learning, 
and  use  technology  to  enable  a multitude  of  methods  through  which  the  processes  can  be  supported. 

Unfortunately,  research  on  the  many  forms  of  computerized  learning  environments  has  been  sparse. 
Empirically-based  design  guidelines  have  rarely  been  offered,  resulting  in  a host  of  diverse  applications  of  technology  but 
not  what  Glaser  (1976)  described  as  a "science  of  design."  The  purposes  of  this  paper  are  to  provide  a brief  overview  of 
learning  environments  and  to  identify  the  foundations  and  underlying  assumptions  of  learning  environments. 

Overview 

The  concept  of  a learning  environment  is  not  new.  Its  roots  can  be  traced  to  early  apprenticeship,  Socratic,  and 
similar  movements  that  have  sought  to  immerse  individuals  in  authentic  learning  experiences,  where  the  meaning  of 
knowledge  and  skills  are  realistically  embedded  (Dewey,  1933;  Pask  & Boyd,  1987).  Recent  perspectives  have  expanded, 
seeking  to  change  the  nature  and  breadth  of  experiences  made  available  to  learners  and  in  the  capacity  to  mediate  these 
experiences  electronically  (Papert,  1993).  Designers  have  operationalized  learning  systems  of  enormous  power  and 
sophistication  based  upon  redefined  notions  of  learner-knowledge  relationships,  contemporary  learning  systems  reflect 
research  and  theory  ranging  from  situated,  contextual  teaching  and  learning  (Brown,  Collins  & Duguid,  1989;  Cognition 
and  Technology  Group  at  Vanderbilt,  1992)  to  cognitive  flexibility  (Spiro,  Fcltovich,  Jacobson,  & Coulson,  1991). 
Comparatively  few  applications,  however,  have  unleashed  the  potential  of  either  the  technologies  or  learners.  A need  to 
optimize  the  capabilities  of  both  emerging  technologies  and  learners  is  apparent.  A clearer  understanding  of  the  roots  of 
alternative  approaches  is  needed. 

Foundations  of  Learning  Environments 

Various  learning  environments  can  be  classified  according  to  the  manner  in  which  they  manifest  their  underlying 
foundations.  Learning  environments  are  rooted  in  five  foundations:  Psychological,  pedagogical,  technological,  cultural, 
and  pragmatic.  All  learning  environments  explicitly  or  tacitly  reflect  these  underlying  models  or  foundations. 

Psychological  foundations  are  rooted  in  beliefs  about  how  individuals  think  and  learn.  Contemporary  learning 
environments  draw  upon  psychological  foundations  from  areas  such  as  constructivism  (Jonassen,  1991),  situated  learning 
(Brown,  Collins,  & Duguid,  1989),  and  cognitive  psychology  (APA,  1992).  Pedagogical  foundations,  on  the  other  hand, 
emphasize  how  knowledge  is  conveyed.  Pedagogical  influences  focus  on  the  methods,  activities,  and  structures  of  the 
learning  environment.  Student-centered  learning  environments  often  draw  upon  pedagogical  approaches  such  as  problem- 
based  contexts  (CTGV,  1992),  and  opportunities  for  experimentation  and  exploration  (Tobin  & Dawson,  1992).  Taken 
together,  psychological  and  pedagogical  foundations  provide  the  basis  for  the  methods  and  strategies  employed,  and  the 
ways  in  which  the  to-be-leamed  content  is  organized. 

When  technological  foundations  are  considered,  they  emphasize  how  the  capabilities  and  limita  ons  of  available 
technology  can  be  optimized  to  create  an  environment  likely  to  engender  the  kinds  of  learning  desired.  Technological 
capabilities  constrain  or  enhance  the  types  of  learner-system  transactions  that  are  possible.  The  challenge  for  dcsignets  is 
to  capitalize  on  the  capabilities  of  emerging  technologies,  while  generating  new  designs  rooted  in  emerging 
psychological  and  pedagogical  research  and  theory. 
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Cultural  foundations  reflect  prevailing  beliefs  about  education,  values  of  the  culture,  and  the  roles  that 
individuals  play  in  society.  In  turn,  they  affect  the  design  of  learning  systems  by  forming  the  underlying  value  systems 
that  guide  design  methodologies.  For  instance,  both  educators  and  society  recognize  the  need  for  educational  system  that 
adequately  meet  the  knowledge  requirements  of  our  rapidly  expanding  technological  society.  Computers  are  prevalent  in 
most  classrooms  and  educational  software  is  widely  available  — schools  are  beginning  to  mirror  the  values  and  priorities 
of  a technological  society. 

Ftagmatic  foundations  reflect  the  practical  constn  ints  of  the  environment.  Each  setting  has  unique  situational 
constraints  that  affect  the  design  of  learning  systems.  Issues  such  as  run-time  requirements,  hardware/software 
availability,  and  financial  concerns  significantly  influence  the  adoption  and  diffusion  of  innovations.  They  emphasize  the 
practical  reasons  a particular  approach  should  or  should  not,  or  can  or  cannot,  be  used  in  a learning  environment. 

The  five  foundations  are  functionally  integrated  in  learning  systems  designs.  The  better  integrated  the 
foundations,  the  greater  probability  of  success  in  the  setting  for  which  it  is  designed.  However,  if  one  or  more  of  the 
foundations  is  not  conceptualized  in  conjunction  with  the  others,  complete  integration  is  not  possible.  For  instance,  an 
environment  rooted  primarily  in  technological  capabilities  (e.g.,  the  use  of  the  Internet  in  the  classroom)  may  be  limited 
in  that  it  excludes  consideration  of  other  foundations;  attention  must  also  focus  on  how  technology  can  support  desired 
thinking,  convey  information,  reflect  cultural  beliefs,  and  meet  pragmatic  requirements.  As  shown  in  Figures  2a  - 2d, 
failure  to  account  for  all  foundations  interacti  vely  causes  predictable  threats  to  the  integrity  of  the  environment. 
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The  underlying  assumptions  determine  how  (or  if)  the  information  within  each  of  the  foundations  will  be 
connected  in  a learning  environment.  This  is  true  regardless  of  specific  biases  and  perspectives.  For  instance, 
objectivists  draw  upon  a subset  of  philosophies,  methods,  and  technological  activities  that  are  uniquely  appropriate  to 
their  underlying  assumptions.  Constructivists  reference  the  same  foundation  pool,  but  derive  distinctly  different 
strategies  based  upon  altered  underlying  assumptions.  The  underlying  assumptions,  then,  dictate  how  the  foundations 
will  be  operationalized  in  any  environment.  As  the  assumptions  vary,  the  foundations,  and  hence  the  features  and 
methods,  of  the  learning  environment  change  accordingly. 

Assumptions  of  Student-Centered  Learning  Environments 

Student-centered  learning  environments  comprise  many  forms,  often  with  few  apparent  similarities.  Isolated 
student-centered  environments  in  science,  mathematics,  social  science,  literature,  and  other  domains  have  prompted 
educators  to  explore  the  structure,  goals,  and  perspectives  of  student-centered  systems.  The  efforts,  however,  often  appear 
dissimilar  in  functions,  goals,  and  features,  thus  making  it  difficult  to  identify  general  design  principles.  Despite  such 
variations,  common  assumptions  have  been  identified  and  are  manifested  either  explicitly  or  implicitly  within  the 
environment  (Hannafin  et  al.,  1994). 

Student-centered  learning  environments  are  rooted  in  several  assumptions.  These  assumptions  are  based  on  a 
synthesis  of  empirical  research,  theory,  and  supporting  examples-  primarily  from  areas  such  as  situated  cognition, 
microworld  design,  mental  model  development,  metacognition,  and  process  learning.  These  assumptions  represent  how 
the  learner,  knowledge,  and  structure  of  the  environment  are  conceptualized.  While  a complete  treatment  of  the 
differences  is  not  possible  here,  several  critical  assumptions  and  accompanying  functions  of  student-centered  learning 
environments  can  be  identified.  Table  1 summarizes  the  assumptions  with  supporting  functions. 


Table  1 : Assumptions  and  Functions  of  Student-Centered  Learning  Environments 


Assumption 

Functions 

Instruction,  as  traditionally 
defined,  is  too  narrow  to  support 
varied  ways  of  promoting  learning 

• Allows  learners  to  "make  sense"  out  of  what  they  know. 

• Supports  meta-knowledge  about  problem  solving. 

• Encourages  deeper  understanding  and  theory  building. 

Activities  must  focus  on  the 
underlying  cognitive  processes- 
not  solely  products  of  learning 

• Increases  meaningful  learning  and  connections  among  ideas. 

• Combats  rote  memory  and  disassociation  of  knowledge. 

• Supports  learning  of  self-regulation  and  meta-knowledge. 

Knowledge  is  dynamic  and 
continuously  evolving 

• Supports  learners  in  building  upon  intuitions  or  mental  models. 

• Understanding  is  refined  through  experience  and  exploration. 

• Addresses  compliance  vs.  evaluation  of  knowledge  issue. 

Individuals  must  assume  a greater 
responsibility  for  their  own 
learning 

• Encourages  unique  sense  making  capabilities  of  learners. 

• Supports  learning  of  self-regulation  skills. 

• Supports  active  learning  and  individual  construction  of 
knowledge. 

Learners  perform  best  when 
varied/multiple  representations  are 
supported 

• Conceptual  diversity  requires  varied  representations  and 
activities. 

• The  potential  for  complex  understanding  increases  as  the 

environment  becomes  rich  and  engaging. 

• Supports  multiple  perspectives  and  flexible  understanding. 

Learning  is  best  when  rooted  in 
relevant  contexts 

• Orients  learners  to  inter-relatedness  of  knowledge. 

• Learner  uses  knowledge  as  a "tool". 

• Cognitive  process  and  context  are  inextricably  tied. 

Learning  is  most  relevant  when 
rooted  in  personal  experience 

• Thinking  originates  from  personal  experience. 

• Normally  abstract  notions  can  be  experienced  and  manipulated. 

• Learners  develop  insights  into  the  "why"  behind  experiences. 
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Table  1.  (continued) 


Assumption 

Function 

Reality  is  not  absolute,  but  is  a 
personal  by-product  of  context, 
interpretation,  and  negotiation. 

• Learners  formulate  and  modify  initial  understanding. 

• Errors  are  useful  as  data  for  refining  understanding. 

• Wisdom  cannot  be  "told." 

Understanding  requires  time 

• Understanding  must  be  cultivated  and  not  described. 

• Understanding  is  deeper  when  learners  "get  to  know"  and 
explore  it. 

• Understanding  transcends  the  information  given. 

Understanding  is  best  supported 
when  processes  are  augmented,  not 
supplanted,  by  technology. 

• Allows  novices  to  become  familiar  with  complex  notions 
without  excessive  cognitive  load. 

• Engages  learners  in  complex  ideas/problems  encountered  by 
experts. 

• Leads  to  understanding  that  surpasses  what  could  be  achieved 
without  support. 

Learners  make,  or  can  be  guided  to 
make,  effective  choices 

• Supports  development  of  learner’s  "need  to  know"  information. 

• Establishes  an  "anchor"  upon  which  further  information  can  be 
added 

• Learners  see  errors  as  a cue  for  further  information  in  process  of 
working  towards  a goal. 

Conslusigns 

Advances  in  technology  have  enabled  the  development  of  a range  of  learning  environments.  These  environments 
reflect  diverging  views  about  the  nature  of  knowledge  and  understanding,  the  role  of  learners,  and  the  manner  in  which 
learning  environments  should  be  structured.  This  paper  has  identified  underlying  assumptions  and  relevant  foundations 
for  a particular  group  of  approaches-student-centered,  computer-enhanced  learning  environments— that  manifest  similar 
features  and  attributes.  Based  upon  the  underlying  beliefs  of  student-centered  learning,  the  value  of  the  environment  can 
be  optimized  based  upon  foundations  that  are  not  only  consistent  with,  but  a direct  manifestation  of,  the  assumptions. 

It  is  important  to  recognize  that  viable  alternatives  to  traditional  instructional  methodologies  exist,  alternatives 
that  are  rooted  in  different  assumptions  and  draw  upon  different  research  and  theory  bases  than  do  traditional  approaches. 
The  shifts  are  fundamental,  not  cosmetic  or  semantic,  in  nature.  The  issue  is  not  simply  one  of  emphasizing  similarities 
across  approaches,  but  comprehending  the  differences  in  assumptions  and  foundations  that  underlie  them.  It  is  unlikely 
that  renaming  traditional  processes,  without  altering  basic  beliefs  about  the  processes  themselves,  will  significantly  alter 
the  nature  of  the  learning  environment.  Indeed,  in  many  cases,  traditional  methods  have  been  largely  unsuccessful  in 
promoting  the  kinds  of  critical  thinking  and  problem-solving  widely  sought.  If  we  aim  to  address  sophisticated  learning 
goals  involving  in-depth  study,  problem  solving,  and  reasoning,  alternative  assumptions,  foundations,  and  methods  must 
be  considered. 
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Abstract 


Screen  designers  for  computer-mediated  instruction  (CMI)  products  must  consider  the  motivational  appeal  of 
their  designs.  Although  learners  may  be  motivated  to  use  CMI  programs  initially  because  of  their  novelty,  this  effect 
wears  off  and  the  instruction  must  stand  on  its  own.  We  surveyed  approximately  92  articles  and  books  to  compile  screen 
design  guidelines  that  were  proposed  by  their  authors  in  whole  or  in  part  on  the  basis  of  their  contribution  to 
motivational  appeal.  Two  primary  types  of  guidelines  were  discovered-those  aimed  at  enhancing  motivation  and  those 
aimed  at  preventing  loss  of  motivation.  The  authors  call  the  first  type  of  guideline  "expansive,"  and  the  second 
"restrictive."  The  authors  propose  a framework  that  designers  can  use  to  apply  the  two  types  of  guidelines  and  arrive  at 
effective  screen  design. 


INTRODUCTION 

Computer-mediated  instruction  (CMI)  systems  are  increasingly  used  for  learning  and  training  in  school  as  well 
as  in  corporate  settings.  Designers  of  CMI  aim  to  design  presentations  that  will  encourage  and  facilitate  learning.  Screen 
design,  as  the  communication  vehicle  for  most  of  the  human-computer  interface,  refers  to  the  purposeful  organization  of 
presentation  stimuli  in  order  to  influence  how  students  process  information  (Hannafin  & Hooper,  1989,  p.  156).  Studies 
on  human  memory  and  message  design  consistently  show  that  well  organized  material  causes  learners  to  develop  and 
maintain  interest  in  lesson  content,  promotes  the  learner  engagement  with  the  material,  and  facilitates  deep  processing  of 
important  information  better  than  does  unorganized  material  (Cook  & Kazlauskas,  1993,  Faiola  & DeBloois,  1988, 
Grabinger  & Albers,  1989;  Hannafin  & Hooper,  1989).  Well  organized  material  also  improves  reading  speed  and 
comprehension,  and  effectiveness  and  efficiency  of  learning  from  CMI  (Galitz,  1989;  Hannafin  & Hooper,  1989; 
Hathaway,  1984;  Strickland  & Poe,  1989). 

CMI  delivers  information  in  a variety  of  ways  including  text,  graphical  images,  color,  animation  and  audio,  and 
other  components  of  screen.  Well  designed  CMI  screens  combine  these  elements  into  a whole,  cohesive  and  balanced 
display  (Moore  & Dwyer,  1994).  Skillfully  designed  CMI  screens  should  not  only  provide  the  content  and  functionality 
to  satisfy  learner's  needs  in  a manner  that  draws  learners'  attention,  motivates  interaction,  and  helps  them  accomplish 
learning  goals  without  confusion  and  fatigue  (Faiola,  1990;  Galitz,  1989;  Jacques,  Preece,  & Carey,  1995,  Shneidei man, 
1992),  but  also  contribute  to  quality  and  usability  (Tufte,  1992). 

Motivation  is  a frequently  used  word  in  the  literature  of  education  and  computers,  however,  the  study  of 
motivation  has  long  been  a neglected  area  in  instructional  technology  (Rezabeck,  1994).  To  be  effective,  instruction 
must  be  appealing  to  learners.  Many  learners  do  find  CMI  programs  novel  and  appealing,  at  least  initially.  Most  CMI 
programs  often  contain  interactive  communication,  automatic  feedback,  animated  graphics,  sound  effects,  and  freedom  to 
make  mistakes  without  fear  of  censure.  If  CMI  screens  are  not  well  designed,  or  lack  motivational  appeal  beyond  the 
novelty  level,  then  learner  involvement  wanes,  and  the  program  is  abandoned  (Keller  & Suzuki,  1988).  Consequently,  it 
is  critical  for  designers  of  CMI  to  make  screens  that  are  appealing  at  a level  beyond  the  novelty  effect  in  addition  to 
making  the  screens  instructionally  efficient  and  effective. 

Keller  (1983)  suggests  that  motivation  attracts  learners  to  the  instruction  and  increases  their  effort  in  relation  to 
the  subject . Bohlin  and  Milheim  (1994)  concur,  stating  that  motivation  provides  appeal  or  interest  for  the  learner  and 
stimulate  learner  effort.  With  reasonable  limits,  the  greater  the  motivation,  the  greater  will  be  the  learning  (Butler,  1972). 
According  to  Keller's  ARCS  model  (Keller,  1987),  instructional  motivation  has  four  main  components:  it  should  gain 
and  sustain  attention  of  learners  (attention),  pertain  to  the  needs  and  familiarity  of  learners  (relevance),  foster  confidence  in 
accomplishing  the  learning  task  (confidence),  and  satisfy  learners  by  meeting  their  expectation  (satisfaction). 

Screen  design  for  effective  CMI  is  not  complete  without  considering  its  motivational  appeal.  Most  designers 
would  agree  with  these  assertions,  however,  many  would  think  of  motivation  for  CMI  as  a consideration  to  be  brought 
up  late  in  the  design  process  (Keller  & Burkman,  1993).  Motivation  should  be  considered  throughout  the  design  and 
development  process  (Okey  & Santiago,  1991;  Keller  & Kopp,  1987).  Many  designers  believe  that  effective  use  of 
motivational  elements  for  CMI  screens  can  contribute  to  increasing  motivation  to  learn.  Keller  and  Burkman  (1993)  go 
so  far  as  to  suggest  that  the  motivation  to  learn  is,  in  large  part,  a CMI  designer's  responsibility,  not  learner's 
responsibility. 

Motivational  screen  design  should  be  related  to  orienting  learners  toward  the  instruction,  encouraging  deep 
processing,  focusing  attention,  engaging  learners  on  task,  and  resulting  in  better  performance  (Grabowski  & Curtis, 

1991 ; Mukherjee  & Edmonds,  1994).  CMI  texts  with  a high  legibility  enable  the  readers  to  devote  their  attention  to 
understanding  whatever  information  the  text  conveys.  Inherently  rewarding  interactions  increase  learner's  intrinsic 
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motivation  to  continue,  which  can  lead  to  a state  of  flow:  a condition  in  which  "people  are  so  involved  in  an  activity  that 
noting  else  seems  to  matter"  (Csikszentmihalyi,  1992,  p.  4). 

The  purpose  of  this  article  is  to  identify  the  available  guidelines  for  effective,  user-friendly  designed  CMI 
software  to  maximize  learners’  performance  of  the  content  material  and  their  motivation  to  learn,  and  place  these 
guidelines  in  a useful  framework  for  making  CMI  system  more  effective,  efficient,  and  appealing. 

GUIDELINES  FOR  MOTIVATIONAL  SCREEN  DESIGN 

The  authors  discovered  two  primary  types  of  guidelines  in  the  books  and  articles  surveyed.  The  first  type  is 
aimed  at  supporting  intrinsic  motivation  or  enhancing  extrinsic  motivation.  The  second  type  is  aimed  at  preventing  the 
loss  or  degradation  of  either  intrinsic  or  extrinsic  motivation.  We  call  the  first  type  of  guideline  "expansive,"  and  the 
second  type  "restrictive." 

There  are  many  components  of  screen  design  which  may  affect  the  motivational  value  of  CMI  screens,  however, 
this  present  study  will  be  focused  on  five  screen  design  elements:  typography,  graphical  images,  color,  animation  and 
audio.  Guidelines  for  each  category  are  listed  based  on  the  expansive  and  restrictive  classifications. 

Typography 

First  of  all,  typography  is  the  key  element  in  text  materials  and  in  almost  any  well-designed  table,  chart,  map, 
or  diagram  (Marcus,  1992).  Generally,  text  on  screens  has  been  found  to  be  less  legible  than  iext  on  paper.  Text  display 
on  computer  screen  can  be  difficult  to  read  or  easy  to  read,  depending  upon  the  font,  the  screen  layout,  and  the  contrast 
provided  (Moore  & Dwyer,  1994). 

Signals  are  writing  devices  that  emphasize  aspects  of  a text’s  content  or  structure  without  adding  to  the  content 
of  the  text  and  help  readers  identify  specific  points  in  a text  (Golding  & Fowler,  1992;  Lorch,  1989).  These  include  titles, 
headings,  pre-  and  post-  instructional  strategies,  such  as  preview,  overview  and  summary,  and  typographical  cues 
(Golding  & Fowler,  1992;  Lorch,  1989).  Although  there  are  many  typographical  motivation  elements  for  CMI  system, 
they  are  all  share  the  goal  of  directing  the  learner's  attention  during  learning.  Signals  including  typographical  cueing  may 

lead  to  enhanced  memory  performance  as  the  result  of  a von  Restorff  effect^.  The  result  is  that  the  isolated  events  have  a 
higher  probability  of  being  recalled  at  a later  time  (Glynn,  1978).  Signaling  is  information  in  text  that  does  not  add  new 
content  on  a topic,  but  that  gives  emphasis  to  certain  aspects  of  the  semantic  content  or  points  out  aspects  of  the 
structure  of  the  content  (Meyer,  1985).  Signaling  should  serve  to  clarify  content. 

Typographical  cueing  system,  defined  as  the  attachment  of  a specific  meaning  to  a part  of  a text  by  displaying  it 
in  a way  which  is  different  from  the  rest  of  the  text,  controls  mathemagenic  behaviors.  Typographic  coding  in  CMI 
screen  is  mainly  used  for  accentuating;  the  accentuation  can  relate  to  single  words,  phrases  or  whole  paragraphs  (van  Nes, 
1986).  The  cueing  systems  guide  the  construction  and  implementation  of  learner’s  prose-processing  decision  criteria 
(Glynn  & Di  Vesta,  1979).  However,  we  should  try  to  establish  a hierarchy  of  importance  using  the  techniques  above  and 
then  employ  it  consistently  throughout  the  CMI  system. 

Expansive  Guidelines 

• Use  graphic  character  fonts  for  attention-catching  because  of  their  size  and  unusual  shape,  even  if  less  legible  than 
normal  fonts  (van  Nes,  1986). 

Restrictive  Guidelines 

• Be  consistent  in  assigning  textual  cues  and  messages  to  the  learner  (Faiola,  1990).  Consistency  in  typographic 
attributes  can  establish  and  convey  a very  clear  visual  message  to  the  viewer  that  (s)he  is  now  reading  a certain 
subject  or  section  of  instructional  content. 

• Use  both  upper  and  lower  case  letters  (Apple  Computer  Inc.,  1989;  Faiola  & DeBloois,  1988).  The  legibility  of  text 
with  letters  in  mixed  case,  i.e.  with  capitals  only  used  for  indicating  the  first  letter  of  a sentence,  a name  etc.,  is 
higher  than  in  upper  case  only  (van  Nes,  1986).  All  upper  case  characters  should  be  used  only  occasionally,  and  for 
the  purposes  of  emphasis  (Strauss,  1991). 


12  von  Restorff  effect  refers  to  "signaled  information  is  remembered  better  because  it  is  perceptually  salient  or  isolated  from 
the  rest  of  the  text."  (Golding  & Fowler,  1992,  p.  100) 
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• Use  high  contrast  between  letters  and  background  to  improve  legibility  and  readability  (Isaacs,  1987;  Pastoor,  1990; 

Reynolds,  1979;  Revlin,  Lewis,  & Davies-Cooper,  1990).  ..  . 

• Left-justify  text,  but  do  not  right-justify  it  (Gamer,  1990).  Limit  text  to  approximately  65  character  per  line,  or  a 
maximum  of  8-10  words  per  line  (Galitz,  1989;  Gamer,  1990).  Increase  the  spacing  between  lines  for  long  lines  of 
text  (Gamer,  1990;  Hartley,  1990). 

• Use  highlighting  techniques  conservatively  and  carefully,  since  they  are  likely  to  attract  the  readers  attention  (App  e 
Computer  Inc.,  1989;  Galitz,  1989;  Hartley,  1987;  Heines,  1984;  Isaacs,  1987;  Rivlin,  Lewis,  & Davies-Copper, 

1 990).  Underlining  and  blinking  should  only  be  used  if  they  do  not  interfere  with  the  legibility  of  the  text  (Heines, 
1984;  Merrill,  1988;  Rivlin,  Lewis,  & Davies-Cooper,  1990;  Tullis,  1988).  Flashing  should  be  reserved  for  items 
that  must  convey  an  urgent  need  for  attention.  Reverse  video  can  be  an  effective  method  in  attracting  the  reader  s 
attention,  but  it  should  be  used  judiciously  to  avoid  the  "crossword-puzzle  effects  (Galitz,  1989,  p.  61). 

• Select  a typeface  with  a simple,  clean  style  and  use  a few  typefaces  in  any  one  screen  or  CMI  program  (Apple 
Computer,  inc.,  1989;  Hartley,  1994).  Most  screens  look  best  with  no  more  than  two  different  fonts,  using  different 
sizes  and  weights  of  each  (Strauss,  1991). 

Graphical  Images 

With  the  advent  of  the  first  widely  used  graphical  user  interface(GUI)  computer  by  Apple  Computer  in  1984,  it 
became  possible  to  do  more  sophisticated  things  with  the  CMI  screen  design  (Jones,  1995).  Graphical  images  including 
pictures  graphics  and  illustrations  throughout  CMI  system  can  function  to  attract  and  maintain  attention,  and  facilitate 
learning  from  text  materials  (Duchastel,  1978;  Levie  & Lentz,  1982;  Rieber,  1994).  Graphical  images  can  also  provide 
functional  guidance,  aesthetics  charm,  and  corporate  or  product  identity  (Marcus,  1992). 

Even  though  using  graphical  images  to  arouse  general  curiosity  and  interest  in  CMI  system  seems  as  a very 
superficial  way,  they  offer  the  potential  to  increase  the  challenge  and  curiosity  of  the  task,  as  well  as  encouraging 
students  to  be  creative  and  use  their  imaginations  (Rieber,  1 994).  Learning  certainly  demands  effort  and  hard  work,  but 
instruction  does  not  need  to  be  boring  and  dull.  So  we  need  deeper  ways  to  maintain  attention  and  interest  beyond  the 
simple  provision  of  interesting  graphical  images.  Screen  designers  attempt  to  achieve  visual  solutions  using  graphical 
elements  that  are  functional,  elegant,  appropriate,  simple  and  economical,  and  consistent  (Hartley,  1987).  The  use  of 
graphical  images  may  enrich  presentations  and  make  them  more  attractive  to  the  learners. 

Expansive  Guidelines 

• Consider  the  overall  standard  of  imagery  that  meet  learner  expectations  for  style  (Rivlin,  Lewis,  Davies-Cooper, 
1990). 

Restrictive  Guidelines 

. Use  simple  and  clear  images  to  make  CMI  more  effective  (Levie  & Lentz,  1982;  Marcus,  1992;  Rivlin,  Lewis,  & 
Davies-Cooper,  1990;  Tompson,  1994).  Don’t  use  images  with  too  much  detail  at  a small  scale  as  this  can  be  lost 
on  screen.  Simple  image  are  more  effective  for  instruction  than  complex  images. 

• Use  graphical  images  for  instructional,  motivational,  or  attention-focusing  effects,  and  not  simply  for  the  sake  of 
including  them  on  the  screen  (Duchastel,  1978,  1983;  Levie  & Lentz,  1982;  Pettersson,  1993;  Rieber,  1994;  Surber 
& Leeder,  1988). 

• Make  sure  all  the  key  components  of  the  graphical  images  are  labeled  (Rivlin,  Lewis,  & Davies-Cooper,  1 990).  Use 
captions  or  title  for  labeling  the  key  elements  of  graphical  images. 

• Consider  the  prior  knowledge  and  cultural  conventions  of  the  learner  in  choosing  graphical  image  components 
(Apple  Computer  Inc.,  1989;  Boling,  Johnson,  & Kirkley,  1994,  Easterby,  1970).  Avoid  sexist,  culturally- 
sensitive,  and  other  potentially  offensive  imagery.  However,  include  pictures  of  people,  plants,  and  animal  with 
relatively  large  and  detail  for  especially  appealing  to  girls,  while  include  vehicles  or  machines  in  action  for  only 
appealing  to  boys  (Jakobsdottir,  Krey,  & Sales,  1994). 

• Obey  any  existing  conventions,  such  as  the  standard  symbols  in  a circuit  diagram  or  the  top  to  bottom  or  left  to 
right  order  for  a low  chart  (Rivlin,  Lewis,  Davies-Cooper,  1990). 
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Color 


Color  can  also  be  used  effectively  within  CMI  system  for  aesthetic  and  motivational  reasons  (Galitz,  1989; 
Hartley,  1987,  1994;  Heines,  1984),  however,  it  is  the  most  sophisticated  and  complex  of  the  visible  language 
components  for  screen  design.  Color  may  promote  deep  processing  of  important  information,  aid  in  organizing  lesson 
content,  allow  reasonable  learner-control  options,  promote  interaction  between  the  learner  and  lesson  content,  and 
facilitate  lesson  navigation  (Hannafin  & Hooper,  1989).  Most  researchers  seem  to  agree  that  colored  images  are  desirable 
from  a motivational  point  of  view,  although  the  research  suggests  that  the  motivation  effect  of  graphical  images  varies 
greatly  with  the  age,  the  intelligence  and  the  education  of  the  reader  (Hartley,  1994).  While  claims  are  often  made  for  the  * 
motivational  value  of  color  in  instructional  materials,  the  research  findings  on  motivation  are  by  themselves  not  strong 
enough  arguments  for  using  color  (Brockmann,  1991;  Misanchuk  & Schwier,  1995). 

. However,  misuse  and  misunderstanding  of  the  use  of  color  is  common.  The  differently  colored  part  will 
involuntarily  attract  fixation  of  the  eye  when  it  scans  the  page.  A part  of  a text  may  rapidly  found  by  giving  it  a specific 
color,  provided  the  learner  knows  which  color  to  look  for.  Color  differences,  therefore,  can  deliberately  be  used  as 
efficient  search  aids  (van  Nes,  1986). 

Expansive  Guidelines 

• Make  color  coding  aesthetically  pleasing  and  logically  toward  lesson  objective  (Chapman,  1993).  Use  a bright  color 
to  cue  the  learner  for  new  information,  while  presenting  the  reminder  of  the  information  in  standard  colors  consistent 
with  the  rest  of  the  screen  (Knupfer,  1995). 

Restrictive  Guidelines 

• Use  color  in  the  conservative  way:  limit  the  number  and  amount  of  colors  used  (Brockmann,  1991 ; Davidoff,  1987; 
Gamer,  1990;  Shneiderman,  1992;  Strauss,  1991).  Use  a maximum  of  five  plus  or  minus  two  colors  per  screen 
(Faiola,  1990;  Marcus,  1992;  van  Nes,  1986). 

• Use  colors  selectively  to  manipulate  attention.  Color  can  be  used  to  highlight  text  or  graphics  to  make  them 
conspicuous  (Durrett  & Trezona,  1992;  Garner,  1990;  van  Nes,  1986). 

• Keep  consistency  in  color  coding  (Faiola,  1990;  Galitz,  1989;  Marcus,  1992).  Consistency  is  crucial  for  CMI  screen 
design.  Carefully  select  colors  for  all  visual  devices  such  as  touch  screens,  buttons,  menus,  and  titles,  and  never 
change  the  coding  scheme  during  the  presentation  (Brockmann,  1991;  Chapman,  1993;  Faiola,  1990;  Faiola  & 
DeBloois,  1988;  Galitz,  1989;  Heines,  1984;  Marcus,  1992;  Milheim  & Lavix,  1992;  Shneiderman,  1992;  Rivlin, 
Lewis,  & Davies-Copper,  1990;  Strauss,  1991). 

• Use  cool,  dark,  low-saturation  colors  (e.g.  olive  green,  gray,  blue,  brown,  dark  purple,  black,  etc.)  for  background 
that  recede  and  do  not  vie  for  the  user’s  attention,  while  foreground  colors  can  be  hotter,  lighter,  and  more  highly- 
saturated  colors  (lemon  yellow,  pink,  orange,  red,  etc.)  that  tend  to  come  forward  on  the  screen  and  attract  the  user's 
eye  (Faiola,  1990;  Faiola  & DeBloois,  1988;  Milheim  & Lavix,  1992;  Strauss,  1991;  Tufte,  1992;  van  Nes,  1986). 

• Avoid  the  use  of  complementary  colors  (e.g.  blue/orange,  red/green,  violet/yellow)  (Heines,  1984). 

• Use  commonly  accepted  colors  for  particular  actions,  remembering  that  such  color  may  be  appropriate  only  for 
specific  culture  or  social  systems  (Shneiderman,  1992).  Colors  should  match  conventional  meaning  and  symbolic 
associations  (Chapman,  1993;  Loosmore,  1994). 

• Use  higher  levels  of  brightness  for  distinguishing  colors  according  to  learners  age  (Misanchuk  & Schwier,  1995). 
Animation/Audio 

Animation  can  be  an  effective  way  of  arousing  and  maintaining  a learner's  attention  during  CMI  system.  The 
purpose  of  the  animation  sometimes  is  not  to  teach  something,  but  only  to  attract  and  focus  the  learner's  attention  onto 
the  CMI  screen  (Rieber,  1990,  1994).  Animation  should  be  incorporated  only  when  its  attributes  are  congruent  to  the 
learning  task  (Rieber,  1990).  Screen  designers  should  not  be  over-ambitious  with  animation.  Moving  a small 
component  on  the  screen  is  often  more  effective  and  generally  easier  to  achieve  (Rivlin,  Lewis,  & Davies-Cooper,  1990). 

Although  animation  can  be  a very  dramatic  visual  effect,  the  efficacy  of  animated  presentations  on  cognition  is 
quite  subtle  (Rieber,  1990).  Animation,  like  any  graphical  images,  would  also  be  expected  to  help  learners  to  visualize  a 
dynamic  process  that  is  difficult  or  impossible  for  them  to  visualize  on  their  own,  and  would  facilitate  learning  tasks 
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(Rieber,  1990).  The  mental  processes  of  selection  and  organization  are  particularly  important  considerations  in  designing 
animated  displays  because  of  the  temporal  nature  of  these  displays. 

Audio  in  CMI  program  is  not  a new  phenomenon,  however,  little  research  has  been  done.  In  language  and 
speech  training  audio  was  also  a common  feature  (Aamtzen,  1993).  Audio  can  draw  and  hold  learners  attention  to  the 
most  important  parts  of  the  CMI  screen,  complement  the  visual  information  on  the  screen,  support  the  learner  reading 
the  text  on  the  screen  (Aamtzen,  1993).  Generally,  there  are  three  kinds  of  audio  sources:  (a)  voice  or  speech,  (b)  music, 

and  (c)  sound  or  natural  effects  (Brewer,  1986).  . 

The  function  of  sound  is  temporal  when  spoken  instructions  and  directions  are  provided  about  a future  event,  or 
feedback  about  the  past,  creating  a synergistic  relationship  with  the  visual  presentation  (Mann,  1995).  Temporal  sound 
may  help  to  focus  learner's  attention  on  critical  information  from  the  CMI  program,  particularly  when  they  can  listen  to 
the  highlights  as  well  as  read  the  details,  however,  Barron  and  Kysilka  (1993)  report  that  digital  audio  did  not  have  a 
significant  effect  on  overall  comprehension  of  the  tutorial  content. 

Expansive  Guidelines 

• Use  graphical  animation  to  explicitly  represent  highly  abstract  and  dynamic  concepts  in  science,  including  time- 
dependent  processes  (Park,  1994;  Park  & Hopkins,  1993;  Reiber,  1990,  1994). 

• Use  animation  as  a substitute  or  aid  for  verbal  communication  (Park,  1994;  Park  & Hopkins,  1993). 

Restrictive  Guidelines 

• Use  animation  sparingly  and  only  when  required  to  enhance  CMI  program  (Rieber,  1990;  Venezky  & Osin,  1991). 
Small  and  simple  animation  may  be  more  effective  than  large  ones  (Rivlin,  Lewis,  & Davies-Cooper,  1990). 

• Use  congruent  animation  to  the  learning  task  (Reiber,  1990,  1994).  Use  animation  as  a visual  analogy  or  cognitive 
anchor  for  the  instruction  of  problem  solving  (Park,  1994;  Park  & Hopkins,  1993).  Use  animation  to  simulate 
functional  behaviors  of  mechanical  or  electronic  systems  and  to  demonstrate  troubleshooting  procedures  (Park,  1994; 
Park  & Hopkins,  1993). 

• Avoid  unnecessary  or  gratuitous  animation  on  the  screen  not  to  distract  (Strauss,  1991).  Do  not  harm  using 
unnecessary  animation.  The  simpler  animation,  the  better  result. 

• Use  voices  or  speech  for  providing  information.  When  speech  is  used  as  the  mainstream  provider  of  information, 
same  text  should  appear  on  the  screen.  According  to  redundancy  theories  (Aamtzen,  1993),  this  will  enhance 
learning. 

• Use  calm  music  to  create  a relaxing  atmosphere;  Use  loud  sound  for  alarm  and  warning  messages  (Aamtzen,  1993). 

INTEGRATIVE  GUIDELINES  FOR  MOTIVATIONAL  SCREEN 

Based  upon  the  results  of  exploring  the  motivational  screen  design  guidelines,  the  issues  in  CMI  screen  design 
are  not  which  fonts  are  inherently  best,  whether  to  double  space  routinely,  or  whether  to  employ  color  or  sound,  but  how 
to  best  to  utilize  those  elements  to  promote  appropriate  learning  processing  (Hannafm  & Hooper,  1989).  Each  of 
motivational  screen  elements  is  not  effective  definite  roles  to  play  in  instructional  media.  The  instructional  roles  of  each 
element  and  the  instructional  conditions  appropriate  for  their  use  are  an  attempt  to  move  us  closer  to  a realization  of  the 
effects  on  integrated  design  factors. 

The  task  of  the  instructional  designer  who  wants  to  make  instruction  both  effective  and  appealing  is  to  link  the 
ideas  of  motivational  design  to  an  instructional  design  theory  (Main,  1993;  Okey  & Santiago,  1991).  Motivational 
screen  design  supports  instructional  strategy.  Instructional  strategy  should  be  sound  in  view  of  content  selection  and 
sequencing  appropriate  for  audience  and  learning  goals  (Reigeluth,  1983).  If  these  conditions  are  not  met,  motivational 
screen  design  can't  fix  the  problems  such  as  engaging  learners  in  irrelevant  instruction  or  demotivating  learners  further 
than  usual.  Instructional  design  and  motivational  design  are  complementary,  not  conflicting  ideas. 


Integration  of  Motivational  Elements 

* According  to  dual-coding  model  on  perception  and  cognition  (Paivio,  1978,  1983;  Bleasdale,  1983),  coding  an 
item  in  two  ways  will  enhance  memory  more  than  coding  an  item  in  one  way  only.  Researches  on  cueing  (Grabinger, 
1989;  Grabinger  & Albers,  1989  ) have  shown  that  learners  learn  best  when  they  are  cued  to  specific  information  in  an 
application.  The  cue  summation  theory  has  also  been  the  focus  of  much  research  into  multichannel  communication. 
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This  principle  of  learning  theory  predicts  that  learning  increases  as  the  number  of  available  cues  or  stimuli  is  increased 
(Barron  & Kysilka,  1993).  Designers  of  CMI  should  consider  the  question  of  how  the  combination  of  visual  and  auditive 
screen  design  elements  should  be  for  motivating  learners  (Aamtzen,  1993). 

Some  of  the  more  substantial  guidelines  for  the  CMI  screen  emerge  from  the  tradition  of  print-based  materials 
(Hartley,  1994;  Tinker,  1964,  1965).  Guidelines  generalized  from  researches  related  to  screen  design  offer  specific 
recommendations  as  to  individual  elements  of  the  CMI  screen  design  such  as  typography,  graphical  images,  color, 
animation/audio,  and  so  forth.  Although  single  element  research,  as  a necessary  first  step  in  understanding  how  to 
combine  elements  into  overall  screen,  is  extremely  important  in  identifying  the  strengths,  weaknesses,  and  potential 
problems  of  using  specific  attributes  on  CMI  screen,  they  often  do  not  address  the  overall  visual  dimension  of  the  CMI 
screen  (Grabinger,  1989;  Haag  & Snetsigner,  1994). 

Motivational  screen  design  elements  should  be  integrative  and  holistic  perspective  as  the  effect  of  multimedia  is 
more  than  just  the  sum  of  its  individual  elements  (McKerlie  & Preece,  1993).  The  combination  of  these  elements  to 
create  an  overall  design,  look,  or  aesthetic  is  one  not  typically  addressed  in  the  current  literature  (Hagg  & Snetsigner, 
1994).  Multi-elements  research,  as  "visual  gestalt  of  a screen"  (Grabinger,  1989,  p.  179),  tends  to  be  more  complex  than 
single-element  research  on  motivational  screen  design  for  effective  CMI  program. 

Motivation  and  achievement  interact  in  ways  that  warrant  a "holistic"  examination  of  strategies  incorporated  into 
CMI  program.  It  should  serve  to  captivate  the  learner's  attention,  promote  feelings  and  expectations  of  success,  improve 
perception  of  control,  increase  positive  attributions  to  effort  and  ability,  and  enhance  self-efficacy  and  foster  achievement 
and  positive  affect  through  social  interaction  (Relan,  1992).  Motivating  students  to  learn  and  become  life-long  learners  is 
an  especially  important  and  difficult  task  for  a teacher  (Bohlin,  Milheim,  & Viechnicki,  1993).  CMI  is  often  very 
motivating  to  students,  especially  when  the  lessons  involve  animated  characters,  voice  capabilities,  and  full-color 
graphics  (Blickhan,  1992).  The  use  of  both  visual  and  audio  to  arouse  the  learners'  attention  is  more  likely  to  increase 
interest  than  the  use  of  visual  or  sound  alone  (Frence  & Vockell,  1994).  Motivation  is  a key  factor  in  all  types  of 
learning. 

OveralLMotivational  Guidelines 

• Use  the  motivational  elements  of  CMI  screen  as  economically  as  possible  (parsimony  principle).  The  economic  use 
of  motivational  CMI  screen  elements  should  be  made  of  screen  display  elements  to  get  the  designer's  message  across 
as  simply  and  clearly  as  possible  (Galitz,  1989;  Gamer,  1990;  Heines,  1984;  Rivlin,  Lewis,  & Davies-Copper, 

1990,  Schwier  & Misanchuk,  1993;  Shneiderman,  1992).  A well  design  screen  should  "...exhibit  no  annoying  or 
distracting  features"  (Strickland  & Poe,  1989,  p.  89).  The  implication  is  that  screen  elements  over  and  above  the 
minimum  required  will  be  experienced  as  annoying  or  distracting. 

• Keep  the  CMI  screen  consistent  in  using  the  motivational  elements  of  CMI  screen  (consistency  principle). 
Consistency  in  cueing  should  be  applied  to  all  usages  of  fonts,  sizes,  spacing,  color,  and  any  other  attribute. 

Visually  pleasing  arrangements  focus  the  perception  of  structure  (Faiola,  1990;  Gamer,  1990).  Consistency  is 
critical  for  CMI  screen  design.  Consistency  states  that  the  appearance,  location,  and  behavior  of  motivational 
elements  should  remain  constant,  and  that  screen  elements  with  similar  function  should  share  similar  appearance, 
location,  and  behavior  (Faiola,  1990;  Galitz,  1989;  Lucas,  1991;  Marcus,  1992;  Shneiderman,  1992).  Consistency 
also  makes  a CMI  program  easy  to  learn,  use,  and  remember  (Shires  & Olszak,  1992),  and  reduces  the  effort  required 
to  learn  new  programs  (Apple  Computer  Inc.,  1992,  Galitz,  1989). 

• Make  the  CMI  screen  aesthetically  (aesthetics  principle).  It  states  that  general  rules  of  visual  composition  (balance, 
symmetry,  unity,  and  harmony)  should  be  regarded  in  CMI  screen  design  (Galitz,  1989;  Garner,  1990;  Misanchuk, 
1992;  Reilly  & Roach,  1986;  Strauss,  1991;  Thompson,  1994).  Aesthetically  pleasing  CMI  can  attract  and  hold  the 
learner's  attention  more  successfully,  and  will  promote  cognitive  learning  better  than,  CMI  screens  constructed 
without  regard  for  aesthetics  (Hagg  & Snetsigner,  1994;  Lucas,  1991;  Misanchuk,  1992;  Pett,  1989;  Reilly  & 
Roach,  1986;  Schwier  & Misanchuk,  1993;  Thompson,  1994).  Well-designed  CMI  screen  can  be  creative  and 
aesthetically  pleasing  (Misanchuk  & Schwier,  1995). 

Based  upon  the  literature  review,  it  can  be  proposed  a framework  that  can  use  to  apply 
the  two  types  of  guidelines  and  arrive  at  effective  screen  design.  Following  is  the 
summary  of  the  framework.  (Table  1) 
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Table  1 

Framework  for  Motivation  in  the  Screen  Design  Process 


Type  of  design 
in  the 

development 

process 

Learner 

activities 

requiring 

motivation 

Screen  design 
issues  most 
relevant  to 
learner 
activities 

Engagement 

goals 

(Jacques,  Preece  & 
Carey,  1995) 

instructional 

apply 

look  and  feel 

encourage 

design 

judge 

consider 

reflect 

metaphor 
production  value 

function 

explore 

interaction 

facilitate 

design 

attend 

comply 

sequencing 
art  direction 

form 

navigate 

layout 

don't  distract 

design 

click/type 
see  and  hear 

text 

graphical  images 
audio 

In  considering  motivation  as  it  relates  to  screen  design  for  CMI,  it  may  be  useful  to  note  that  the  learners' 
motivation  in  the  experience  is  not  only  important  at  the  highest  level,  which  is  in  relation  to  the  learning.  The  learner 
needs  motivation  for  different  types  of  activities  within  the  CMI  experience:  to  click  a "Forward"  button,  to  explore 
more  then  one  path  in  a hypertext  environment,  or  to  compare  two  video  sequences  at  the  content  level  in  order  to  draw  a 
conclusion  about  some  subject.  Only  one  of  these  activities  represents  what  might  be  called  a "learning  activity," 
although  the  others  must  be  recognized  as  contributors,  and  perhaps  prerequisites,  for  learning  to  occur.  Column  2 of 
Table  1 shows  examples  of  the  types  of  activities  learners  may  carry  out  during  the  CMI  experience,  and  it  breaks  those 
activities  into  three  groups  according  to  the  relative  complexity  of  carrying  them  out. 

For  the  sake  of  this  discussion,  instructional  design  is  presumed  to  mean  the  selection  of  strategies  and  tactics 
for  delivering  the  CMI  experience.  Functional  design  is  the  description  of  the  capabilities  that  a CMI  program  will  have, 
and  the  capabilities  it  will  place  under  control  of  the  program  user,  or  learner.  Forms  design  is  the  creation  or  selection 
of  specific  representations  of  function.  By  way  of  a simplistic  example,  the  instructional  design  might  specify  that 
content  will  be  delivered  in  an  exploratory  environment.  The  corresponding  functional  design  might  specify  that 
"learners  will  be  able  to  collect  items  of  information  and  compare  them  as  they  move  through  the  environment."  These 
functions  would  be  operationalized  as  forms  - perhaps  an  illustrated  knapsack  into  which  the  learner  might  drag  items 
using  a cursor  in  the  shape  of  a tiny  hand. 

Screen  design  issues  exist  within  the  instructional  design,  function  design,  and  forms  design  processes,  although 
most  guidelines  and  discussions  purporting  to  relate  to  screen  design  are  typically  restricted  to  the  forms  design  level. 

For  any  given  element  in  the  overall  screen  design.  Column  3 of  Table  1 lists  examples  of  the  types  of  issues  for  wl  ich 
a screen  designer  is  responsible,  and  divides  them  into  categories  corresponding  to  the  learners'  activities. 

The  authors  feel  that  once  the  distinctions  we  have  drawn  between  levels  of  screen  design  issues  are  clear,  it  may 
be  possible  to  characterize  the  overarching  principle  guiding  each  level  of  design  activity  according  to  the  three  primary 
goals  of  engagement  laid  out  by  Jacques,  Preece,  and  Carey  (1995)  in  their  recent  discussion  of  engagement  as  a design 
concept  for  multimedia.  Although  the  design  of  an  individual  on-screen  navigation  button  might  aim  to  encourage 
learners  by  using  animation  to  attract  their  attention  and  audio  to  entice  them  into  choosing  the  button,  it  is  our  opinion 
that  any  "encouraging"  features  of  a navigation  button  should  be  subordinated  to  the  "don't  distract"  principle.  At  the  top 
of  the  chart,  however,  a screen  designer  may  wish  to  be  cautious  regarding  the  possible  distractions  inherent  in  a chosen 
metaphor,  but  that  designer  should  err  on  the  side  of  "encouragement"  when  looking  for  guidance  at  this  level  of  the 
design. 

Our  survey  of  design  guidelines  reveals  that  most  guidelines  fall  into  what  we  have  called  the  "restrictive"  category', 
and  further  that  most  of  them  address  only  the  bottom  two  rows  shown  in  Table  1 , with  the  majority  addressing  the  lowest 
level  only.  We  speculate  that  this  may  account  for  an  overall  "restrictive"  view  of  screen  design  in  the  field,  and  that  this 
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view  may  keep  instructional  designers  from  discovering  the  strategies  used  by  graphic  designers,  multimedia  designers,  and 
others  in  related  disciplines  to  enhance  the  motivational  aspects  of  screen  design. 

CONCLUDING  REMARKS 

Based  upon  the  increasing  use  of  CMI  program  for  both  education  and  corporate  training,  screen  design  is 
becoming  an  issue  of  great  importance  for  instructional  designers  and  developers  in  many  different  content  areas. 
Instructional  screen  must  provide  effective  and  efficient  instruction,  appropriate  navigation  tools,  and  aesthetics.  This 
article  has  sought  to  provide  an  integrated  list  of  motivational  screen  design  guidelines  for  effective  CMI  systems.  It  is 
hoped  that  these  guidelines  will  provide  a useful  framework  for  making  CMI  system  more  effective,  efficient,  and 
appealing.  Implications  for  motivational  screen  design  will  also  help  instructional  developers  for  practical  use.  Without 
better  understanding  of  motivational  screen  design  guidelines,  efforts  for  designing  CMI  system  will  continue  largely  by 
trial-and-error.  In  short,  CMI  designer  should  focus  on  motivational  screen  design  that  make  the  learning  more  appealing 
and  the  learner  more  confident. 
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Abstract 


Evaluating  and  selecting  the  appropriate  software  is  a very  important  component  of  success  for  using 
multimedia  systems  in  both  educational  and  corporate  settings.  This  paper  attempts  to  explore  the  issues  surrounding 
the  evaluation  of  multimedia  software  and  to  present  a comprehensive  criteria  for  evaluating  and  selecting  multimedia 
software  for  effective  instruction,  from  general  software  evaluation  criteria  to  the  specialized  multimedia  evaluation 
criteria.  An  integrated  checklist  for  multimedia  software  evaluation  was  included.  ( Key  Words:  Computer-Mediated 
Multimedia  (CMM),  Software,  Criteria  Selection,  Evaluation,  Instruction) 

. INTRODUCTION 

The  use  of  microcomputers  for  education  and  training  in  schools  and  homes  as  well  as  in  corporate  settings  are 
commonplace  since  relatively  inexpensive  microcomputers  came  on  the  market  as  an  instructional  tool.  Many 
educational  professionals  believe  that  in  the  world  of  computers,  media,  and  digital  technologies,  multimedia  would  play 
a large  part  in  educational  reform  (Galbreath,  1992;  Jost  & Schneberger,  1994),  and  the  software  available  for  computer- 

mediated  multimedia  (CMM)13  has  been  increasing  both  in  quantity  and  in  quality  over  the  past  couple  of  years. 

Few  instructional  software  programs,  however,  are  field  tested  with  actual  students  prior  to  distribution  (Dudley- 
Marling  & Owston,  1987;  Heller,  1991).  The  ways  in  which  these  software  are  used  vary  with  the  context  of  their  use, 
with  different  age  levels,  subject  areas  or  classroom  setting.  Observers  of  instructional  software  have  cited  a number  of 
specific  problems  with  courseware,  including  technical  inadequacy,  poor  pedagogy,  amateurish  programming,  and 
inadequate  documentation  (Heller,  1991). 

CMM  system  is  an  entirely  new  kind  of  media  experience  bom  from  TV  and  computer  technologies.  It  can  be 
defined  as  the  integration  of  two  or  more  communication  media,  controlled  or  manipulated  by  the  user  via  a computer,  to 
present  information  (Cotton  & Oliver,  1993;  Holsinger,  1994;  Galbreath,  1992;  Poncelet  & Proctor,  1993;  Tolhurst, 
1995).  CMM  program  is  a powerful  combination  of  text,  images,  animation,  sound,  color,  and  video  in  a single, 
computer-controlled  presentation.  CMM  entails  "the  use  of  the  computer  to  integrate  and  control  electronic  media  devices 
such  as  monitors,  videodisc  players,  CD-ROM  players,  and  other  electronic  equipment"  (Poncelet  & Proctor,  1993,  p. 

93),  and  offers  random  access.  Evaluation  and  Selection  of  CMM  systems,  therefore,  may  be  different  from  computer- 
mediated  instruction  (CMI). 

Instructors,  as  well  as  parents,  often  have  difficulty  distinguishing  quality  instructional  software  from  the  trivial 
and  ineffective  software  now  on  the  market.  Guidelines  and  sources  of  knowledge  about  evaluating  and  selecting  quality 
software  program  are  limited  (Chang  & Osguthorpe,  1987).  As  more  of  the  instructional  software  is  placed  on  the 
market,  the  need  for  careful  review  of  the  material  prior  to  its  purchase  becomes  increasingly  necessary.  CMM  software, 
like  all  other  educational  material,  should  be  evaluated  with  a thorough  and  detailed  evaluation  before  it  is  used  in  the 
classroom  (Heller,  1991). 

According  to  the  report  of  Educational  Products  Information  Exchange  (EPIE)  (Taylor,  1987),  only  about  a 
quarter  of  available  software  for  elementary  and  secondary  schools  have  been  evaluated  adequately.  Neill  and  Neill  (1993) 
also  report  that  only  7.7%  of  the  total  reviewed  software  as  'worth  of  an  A grade'  using  their  criteria  for  quality  assurance. 
This  means  that  the  end-users  of  computer  softwares  would  have  to  conduct  its  own  evaluation  of  three-quarters  of  the 
instructional  software  program  it  is  considering.  There  appears  to  be  a continuing  need,  therefore,  for  an  approach  to 
evaluating  software  which  is  general  enough  to  be  widely  applicable  but  specific  enough  to  provide  the  kind  for 
information  that  will  allow  decisions  to  be  made  about  the  acceptability  of  software  programs  under  reasonable 
consideration. 

The  purpose  of  this  paper  is  to  explore  the  issues  surrounding  the  evaluation  of  multimedia  software  and  to 
present  a comprehensive  criteria  for  evaluating  and  selecting  CMM  software  for  effective  instruction,  from  general 
software  evaluation  criteria  to  the  specialized  multimedia  evaluation  criteria. 


13  Computer  mediated  multimedia  systems  incorpoate  "the  computer  as  a display  device,  management  tool,  and/or  source  of 
text,  pictures,  graphics,  and  sound"  ( Heinich,  Molenda,  Russell,  & Smaldino,  1996,  p.  260). 
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EVALUATION  AND  SELECTION  CRITERIA 


Evaluation  assumes  a value  judgment.  When  evaluating  and  selecting  an  instructional  software,  there  are  some 
general  principles  that  apply  to  all  categories  of  software.  Almost  every  product-oriented  evaluation  includes  some  items 
about  the  use  of  the  program.  These  aspects  considered  for  evaluating  instructional  software  might  include  such  things  as 
contents,  curriculum  issues,  design  factors,  technology  factors,  human  factors,  documentation  and  packaging, 
availability  of  support  materials,  classroom  management,  vendor  factors,  cost  factors,  and  so  on  (Gray,  1991;  Gros  & 
Spector,  1994;  Heller,  1991;  Knight,  1992;  Reiser  & Dick,  1990;  Tolhurst,  1992). 

Even  though  most  of  these  checklist  items  were  created  for  evaluating  traditional  CMI,  similar  proposals  have 
been  made  for  the  evaluation  of  CMM  software  (Atkins,  1993;  Gros  & Spector,  1994;  Hutchings  et  al.,  1992;  Knight, 
1992).  CMM  software  is  quite  different  from  traditional  CMI  in  terms  of  linking  display  tools,  management  tool,  and/or 
design  elements  such  as  text,  graphical  images,  animation  and  sound.  For  this  reason,  new  items  of  evaluation  should  be 
added.  Nielsen  (1990)  proposed  five  additional  parameters  for  evaluating  the  usability  of  multimedia  software:  ease  of 
initial  learning,  efficiency  of  use,  ease  of  remembering  interface  items,  error  rates,  and  subject  response  to  the  system. 

Well  designed  CMM  software  is  instructionally  sound  materials  that  take  advantage  of  the  unique  capabilities  of 
the  computer.  Unfortunately,  however,  not  all  programs  marketed  as  CMM  are  well-designed,  instructionally  sound,  and 
technically  reliable  (Sloane,  Gordon,  Gunn,  & Mickelsen,  1989).  To  make  a reasonable  decision,  criteria  should  be 
developed  to  give  the  evaluator  a framework  for  the  task.  The  most  important  criterion  for  CMM  software  evaluation  is 
the  consideration  which  reflects  the  needs  and  characteristics  of  individual  teachers  and  students  in  using  the  CMM 
software. 

Teachers  can  use  a formal  evaluation  of  CMM  software  to  guide  their  initial  screening  of  software,  but  their 
final  judgment  should  depend  upon  their  own  observations  of  their  students  as  the  students  interact  with  the  software 
(Dudley-Marling  & Owston,  1987).  There  is  no  one  best  CMM  software-different  software  works  best  for  different 
learners  and  for  different  instructional  purposes,  and  no  one  CMM  product  is  ideal  for  any  location  that  serves  diverse 
learners  (Sorge,  Campell,  & Russell,  1993).  Criteria  considered  for  evaluating  CMM  software  in  the  study  include 
content  issues,  instructional  design,  user  interface,  and  documentation.  Evaluation  criteria  include  address  both  general 
CMI  evaluation  criteria  and  the  specialized  CMM  evaluation  criteria. 

Content  Criteria 

The  content  presented  in  CMM  software  should  be  appropriate  to  the  its  objectives  and  ictlect  the  level  of 
difficulty,  sequencing  and  quantity  of  content  sufficient  to  meet  those  objectives.  Because  objectives  are  used  to  plan 
instruction,  to  facilitate  the  evaluation  of  course  outcomes,  and  to  prepare  and  review  test  items  (Taylor,  1987). 
Objectives  may  affect  certain  learner  characteristics.  The  target  audience,  therefore,  should  be  specified,  grade  level  and 
particular  characteristics  such  as  gifted,  talented,  or  handicapped  (Gold,  1984).  The  learning  objectives  should  be 
delineated,  including  how  the  various  features  of  the  instructional  design  fulfill  the  learning  objectives,  and  should  be 
stated  in  measurable  terms  so  that  the  evaluation  of  lesson  will  reveal  whether  the  mastery  has  been  achieved. 

The  contents  and  instructions  of  CMM  software  should  be  accurate,  fair,  and  valued.  The  presentation  of  CMM 
system  content  should  not  only  be  clear  and  logical,  but  be  sufficiently  simple,  complex,  technical  or  non-technical  for 
the  intended  target  learners  (Bitter  <&  Camuse,  1984;  Gold,  1984). 

General  Criteria 

• Instructional  objectives  are  clearly  stated. 

• Definitions  of  target  audience  and  prerequisite  skills  are  stated. 

• Contents  and  vocabulary  level  for  intended  users  are  appropriate. 

• Specific  information  presented  is  accurate  and  answers  are  correct. 

• Sequence  of  lesson  information  and  instructions  are  logical  and  clear. 

• Contents  of  cultural,  sexual,  and  other  stereotypes  are  free. 

Specialized  MultimediaJCxiteria 

• Contents  of  text,  image,  animation,  sound,  video,  etc.  are  congruent  to  CMM  information. 

• Terms  or  words  in  the  CMM  software  are  chosen  appropriate  for  linking  multimedia  design  elements  (Tolhurst, 
1992). 
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Instructional  Design  Criteria 

Interaction  activities  in  educational  setting  not  only  maintain  learners'  attention  and  increase  their  involvement 
on  learning  task,  but  also  result  in  better  performance  on  knowledge  and/or  skills.  One  of  the  essential  features  of  CMM 
software  in  contrast  to  some  other  instructional  media  is  its  capacity  to  require  and  act  upon  learners'  interaction. 
Researchers  emphasize  this  important  aspect  of  CMI,  however,  many  commercial  CMI  software  fall  short  in  the 
characteristics  (Alessi  & Trollip,  1991).  Designing  interactions  which  are  frequent,  relevant,  and  increase  learning  is 
harder  than  even  experienced  CMM  developers  believe.  Software  evaluator  should  consider  learner  involvement  as  an 
important  for  evaluating  CMM  software. 

Another  advantage  of  CMM  is  its  capability  to  individualize  students’  learning.  The  ability  of  instructional 
programs  to  adapt  to  individual  needs  rests  upon  the  type  of  individual  diagnosis  they  are  capable  of  doing.  Adaptive 
instruction  must  be  capable  of:  (a)  gathering  diagnosis  information;  (b)  inferring  from  this  information  the  specific  needs 
of  the  students,  and  (c)  adjusting  instruction  accordingly  (Venezky  & Osin,  1991).  Good  CMM  software  will  adapt  to 
the  learner,  capitalizing  upon  his/her  talents,  giving  extra  help  where  the  learners  is  weak,  and  providing  motivators  each 
learner  responds  to.  However,  most  commercial  software  works  about  the  same  for  all  learners.  Matching  learners  up 
with  appropriate  lessons  and  methodologies  is  important  design  factor  of  CMM  software. 

Motivation  is  an  essential  factor  of  instruction  and  learning.  Instructional  designers,  therefore,  should  use 
appropriate  motivational  strategies  for  age  level,  and  social  and/or  cultural  background  of  the  learner.  Several  motivation 
researches  on  CMI  support  that  CMM  techniques  enhance  learners'  motivation  and  result  in  better  learning.  For  the 
design  of  CMM  programs,  designers  should  first  judge  the  degree  of  intrinsic  motivation  a course  might  tap  in  the  target 
audience,  and  then  decide  how  much,  if  any,  extrinsic  motivation  is  needed.  The  options  for  extrinsic  motivators  usually 
include  game  formats,  humor,  use  of  personal  name  for  variation  in  drill  formats  and  feedback  on  progress  (Venezky  & 
Osin,  1991). 

Feedback  is  the  reaction  of  a CMM  software  to  the  learners'  response  and  may  take  many  forms,  including  text 
message  and  graphic  illustration.  Its  primary  function  is  to  inform  the  learner  of  the  appropriateness  of  a response.  There 
are  two  kinds  of  feedbacks:  informative  and  motivational  feedback.  Informative  feedback  enables  the  learner  to  take 
corrective  actions  regarding  behavior  deficits,  while  motivational  feedback  is  to  increase  the  likelihood  that  the  learner 
will  continue  to  emit  the  behavior.  Thus,  motivational  feedback  is  a form  of  reinforcement  and  most  CMI  researches  on 
feedback  support  the  effectiveness  of  feedback  on  learning  (Taylor,  1987).  To  be  a good  CMM  software,  it  can  generate, 
store,  and  utilize  appropriate  feedback  about  learner  needs. 

General  Criteria 

• Learners  are  actively  involved  with  interaction. 

• Availability  of  varying  levels  of  difficulty  is  provided  according  to  the  skill  level  of  the  learner. 

• Motivational  factors  are  included  in  CMM  software. 

• Appropriate  feedback  is  provided. 

Specialized  Multimedia  Criteria 

• Learner  can  stop  and  move  to  different  places  in  the  CMM  program  without  repetition. 

• CMM  software  encourages  a guided  discovery  or  discovery  mode  of  learning  (Tolhurst,  1992). 

User  Interface  Criteria 

One  of  the  claims  about  CMM  software  is  that  the  user  is  free  to  explore  paths  through  the  combinational 
information  such  as  text,  graphics,  sound,  animation,  video,  etc.  under  computer  control.  The  main  concern  of  user 
interface  criteria  is  always  for  an  easy-to-use,  easy-to-install,  and  easy-to-transport  system  (Knight,  1992).  The  evaluation 
of  a piece  of  CMM  software  components  can  be  guided  by  these  inquiries  topics:  ease  of  use,  clarity  of  directions  and 
responses,  simple  error  handling,  screen  design,  learner  control,  and  record  keeping  (Chang  & Osguthorpe,  1987;  DeJoy 
& Mills,  1989;  Heller,  1991;  Knight,  1992;  Tolhurst,  1992). 

CMM  software  installation  should  not  require  the  service  expert,  but  be  able  to  be  done  on  a hard  disk.  The 
steps  should  be  clear  and  well  defined.  Tutorials  should  be  included  to  aid  the  quick  mastery  of  how  the  software  works. 
One  should  be  able  to  learn  the  basics  in  a short  time.  The  CMM  software  needs  to  be  branched  in  such  a way  as  to 
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allow  sophisticated  users  to  skip  over  the  basic  directions.  After  mastering  the  mechanics  of  using  the  CMM  software,  a 
learner  will  want  to  be  able  to  streamline  the  commands.  So  frequent  users  can  use  shortcut  such  as  abbreviations,  special 
keys,  hidden  commands,  macro  facilities,  and  so  on.  Consistent  sequences  of  actions  should  be  required  in  similar 
situations  and  consistent  commands  should  be  employed  throughout  (Shneiderman,  1992). 

Directions  on  how  to  operate  the  CMM  software  should  be  given  in  language  appropriate  to  the  target  audience. 
Language,  sentence  length  and  complexity,  and  style  of  communication  to  the  user  should  be  clear  and  appropriate  to  the 
capabilities  of  the  intended  audience.  The  required  reading  level  of  the  intended  user  must  be  respected  (Gold,  1984). 

Basically,  there  is  lack  of  systems  error  in  CMM  software,  however,  learners  occasionally  hit  the  wrong  key(s) 
by  mistake.  Even  designing  error-free  system  is  best,  it  is  very  difficult  job.  If  an  error  is  made,  it  is  hoped  that  nothing 
harmful,  such  as  losing  one's  data,  will  occur.  System  should  detect  the  error  and  offer  simple,  comprehensive 
mechanisms  for  handling  the  error  (Shneiderman,  1992).  The  user  should  be  signaled  when  a wrong  key  is  hit.  The 
learner  can  repair  only  the  faulty  part  instead  of  retyping  the  entire  command.  Erroneous  commands  should  leave  the 
CMM  system  state  unchanged,  or  the  system  should  give  instruction  about  restoring  the  state. 

It  is  essential  that  the  instructional  information  be  formatted  on  the  CMM  screen  for  easy  reading.  Its  display 
should  keep  clean,  simple,  attractive,  and  aesthetic  balance  using  screen  design  elements  such  as  text,  image,  animation, 
sound,  color,  video,  etc.  The  use  of  visual  cues  can  be  effective  in  gaining  and  keeping  the  learners'  attention  during 
instruction  (Faiola  & DeBloois,  1988;  Poncelet  & Proctor,  1993).  The  use  of  screen  design  elements  should  be 
appropriate  to  CMM  software  and  enhance  the  learning  process  and  results. 

Learner  control  is  a crucial  design  variable  in  all  CMI  software.  This  means  whether  control  of  sequence, 
content,  methodology,  and  other  instructional  factors  is  determined  by  the  learners,  the  lesson,  or  some  combination  of 
two  (Alessi  & Trollip,  1991).  In  reality,  all  lessons  have  a mixture  of  learners  and  lesson  control.  The  whole  notion  of 
interactivity  is  realized  when  course  allows  the  learners  to  weave  his/her  own  educational  environment.  This  can  be  done 
by  granting  the  learner  control  over  certain  aspects  of  the  CMM  software. 

If  CMM  software  is  to  utilize  the  full  capabilities  of  computer  effectively,  it  should  have  a sophisticated 
management  systems.  One  of  the  most  unique  and  useful  capabilities  of  CMM  software  is  its  ability  to  keep  track 
automatically  of  learner  progress  through  software  materials.  If  CMM  software  is  equipped  with  this  capacity,  it  will  be 
much  benefit  for  teachers  (Truett  & Gillespie,  1984).  Good  record  keeping  abilities  should  allow  for  the  tracking  of  an 
entire  class  of  learners,  not  merely  the  accounting  of  one  learner's  work. 

General  Criteria 

* Management  system  of  CMM  software  is  easy  to  use  and  flexible. 

* Screen  displays  are  easy  to  read  (text  size/color/spacing). 

* Screen  design  elements  such  as  text,  image,  sound,  animation  and  color  combine  to  enhance,  not  distract  from 

information  presentation. 

* CMM  software  allows  the  learner  to  correct  his/her  error.  Error  messages  are  helpful  and  user-friendly. 

* Screen  display  should  be  kept  clean,  simple,  attractive  as  well  as  aesthetically  balanced.  Special  effects  are  used 

effectively  and  not  overdone. 

• Learners  have  control  over  the  rate  of  presentation  and/or  navigation. 

• A clear  and  useful  summary  of  learners'  activities  and  progress  is  provided. 

Specialized  Multimedia  Criteria 

• Multimedia  element  links  and  system  links  are  distinguishable  (Tolhurst,  1992). 

• CMM  software  contains  different  contexts  of  use  (e.  g.  descriptive  text  and  a glossary). 

• CMM  software  contains  aids  to  assist  learner  navigation. 

Documentation  Criteria 

Documentation  is  to  help  the  user  install  and  maintain  the  CMM  software.  The  purpose  of  user’s  manual  is  to 
inform  the  learners  of  two  things:  the  operation  of  the  CMM  software  and  the  instruction  on  software  design  (Gold, 
1984).  User's  manual  should  be  included  the  following:  (a)  specifications  of  the  hardware  configuration,  operating 
system  and  programming  language  code;  (b)  definition  of  any  external  software  required;  (c)  installation  instructions;  (d) 
instructions  on  how  to  operate  the  software;  and  (e)  explanation  of  how  to  exercise  features. 
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The  instruction  on  CMM  software  design  should  be  written  objectives,  content,  curriculum  issues,  and 
evaluation.  The  target  audience  should  be  specified  as  to  age,  grade  level,  any  particular  characteristics  such  as  gifted  or 
handicapped.  For  the  knowledge  issues,  the  knowledge  and/or  skill  prerequisite  for  the  CMM  software  must  be  clearly 
stated.  For  evaluation,  the  CMM  software  should  provide  the  teacher  and  the  student  a way  to  evaluate  learner 
performance  and  effectiveness  (Gold,  1984). 

It  is  imperative  that  CMM  software  be  accompanied  by  a technical  manual  containing  general  information, 
functional  description,  and  required  features  (Gold,  1984).  The  technical  manual  should  include  general  information  that 
describes  the  technical  features,  hardware  configuration,  operating  system  and  programming  language  or  code.  It  should 
also  contain  complete  installation  and  start-up  instructions.  The  manual  should  contain  a functional  description  including 
an  explanation  of  how  each  feature  works,  when  it  operates,  under  what  conditions  and  any  restrictions  that  apply  to  its 
use.  It  should  explain  in  detail  how  to  exercise  each  feature. 

General  Criteria 

• User’s  manual  contain  Retailed  and  complete  indices  of  the  information  available  in  the  CMM  software. 

• Commands  for  operation  are  consistent  and  thorough,  including  the  use  of  multiple  diskette  or  discs  (DeJoy  & 

Mills,  1989). 

• Directions  for  installation  and  operation  are  clear,  accurate  and  complete  (Sorge,  Campell,  & Russell,  1993). 

• Specifications  of  hardware  requirements,  operating  system  and  programming  language  code  are  provided. 

• Any  requirements  for  staff  support  are  made  clear. 

• An  information  ’hot  line'  is  available. 

Specialized  Multimedia  Criteria 

• The  linking  maps  or  descriptions  of  the  hypertext  and/or  other  screen  links  are  provided  (Tolhurst,  1992). 

• Any  identification  or  description  of  the  branching  techniques  used  in  CMM  software  is  presented. 

CONCLUDING  REMARKS 

The  focus  of  this  presentation  is  on  discussing  the  issues  surrounding  the  evaluation  of  CMM  software  and  on 
building  a comprehensive  criteria  for  evaluating  and  selecting  CMM  software  for  effective  instruction.  These  criteria  will 
provide  a useful  framework  to  help  educators  and/or  trainers  select  a quality  instructional  software  for  their  instructional 
purposes.  However,  there  is  no  widely  agreed-upon  standards  or  criteria  for  CMM  software  evaluation.  It  is  a paramount 
need  to  develop  not  just  minimal  standards  but  standards  that  will  act  as  guidelines  to  help  developers,  evaluators,  and 
consumers  determine  what  quality  CMM  softwares  are. 
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Appendix 

Multimedia  Evaluation  Checklist 

General  Information  Date: 


Software  Title: 

Source/Distributor: 

Phone:  Fax: 

Subject  Area: 

Cost: 

Target  Audience: 

Length: 

Computer  Platform:  ( ) Macintosh  (II,  lie,  Ilgs) 

( ) IBM  ( ) Other 

Memory:  ( )1M  ( ) 2M  ( ) 3M  ( ) 4M  ( ) 5M  ( ) 10  M ( ) Other  ( ). 

Content 


Criteria 

Rating* 

Weight** 

General 

• Instructional  objectives  are  clearly  stated. 

• Definitions  of  target  audience  and  prerequisite  skills  are  stated. 

• Contents  and  vocabulary  level  for  intended  users  are  appropriate. 

• Specific  information  presented  is  accurate  and  answers  are  correct. 

• Sequence  of  lesson  information  and  instructions  are  logical  and  clear. 

• Contents  of  cultural,  sexual,  and  other  stereotypes  are  free. 

Multimedia 

• Contents  of  text,  image,  animation,  sound,  video,  etc.  are  congruent  to 
CMM  information. 

Z Terms  or  words  in  the  software  are  chosen  appropriately  for  linking 
multimedia  design  elements. 
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Instructional  Design  Criteria 


Criteria 

Rating* 

Weight** 

General 

• Learners  are  actively  involved  with  learner  interaction. 

• Availability  of  varying  levels  of  difficulty  is  provided  according  to  the 
skill  level  of  the  learner. 

• Motivational  factors  are  included  in  software. 

• Appropriate  feedback  is  provided. 

Multimedia 

X Learner  can  stop  and  move  to  different  places  in  the  program  without 
repetition. 

X Software  encourages  a guided  discovery  or  discovery  mode  of  learning. 

User  Interface  Criteria 


Criteria 

Rating* 

Weight** 

General 

• Management  system  of  the  software  is  easy  to  use  and  flexible. 

• Screen  displays  are  easy  to  read  (text  size/color/spacing). 

• Screen  design  elements  such  as  text,  image,  sound,  animation  and 
color  combine  to  enhance,  not  distract  from  information  presentation. 

• Software  allows  the  learner  to  correct  his/her  error.  Error  messages  are 
helpful  and  user-friendly. 

• Screen  display  should  be  kept  clean,  simple,  attractive  as  well  as 
aesthetically  balanced.  Special  effects  are  used  effectively  and  not 
overdone. 

• Learners  have  control  over  the  rate  of  presentation  and/or  navigation. 
X A clear  and  useful  summary  of  learners’  activities  and  progress  is 

provided. 

Multimedia 

• Multimedia  element  links  and  system  links  are  distinguishable. 

X Software  contains  different  contexts  of  use  (e.g.  descriptive  text  and  a 
glossary). 

X Software  contains  aids  to  assist  learner  navigation. 

O 
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Document  Criteria 


1 Criteria 

Rating* 

Weight** 

General 

• User's  manual  contain  detailed  and  complete  indices  of  the  information 
available  in  the  software. 

• Commands  for  operation  are  consistent  and  thorough,  including  the  use 
of  multiple  diskette  or  discs. 

• Directions  for  installation  and  operation  are  clear,  accurate  and 
complete. 

• Specifications  of  hardware  requirements,  operating  system  and 
programming  language  code  are  provided. 

• Any  requirements  for  staff  support  are  made  clear. 

• An  information  ’hotline'  is  available. 

Multimedia 

• The  maps  or  descriptions  of  the  hypertext  and/or  other  screen  links  are 
provided. 

X Any  identification  or  description  of  the  branching  techniques  used  in  the 
software  is  presented. 

* Rating:  3 - High  2 - Medium  1 - Low 

**  Weight:  3 - Very  Important  2 - Important  1 - Little  Important 


Overall  Evaluation  and  Recommendations 


Evaluator  Record 
Name: 


Signature: 


Affiliation: 
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Introduction 

One  of  the  most  important  goals  of  instructional  design  is  to  help  learners  to  develop  capabilities  to  solve 
situated  real  life  problems.  Students  should  not  only  be  able  to  memorize  instructional  contents,  but  also  be  able  to  make 
inferences  and  geneiate  problem  solving  solutions  in  novel  and  complex  situations.  Traditionally,  it  is  believed  that  if  we 
can  help  students  to  learn  abstract  rules  and  principles,  they  will  be  able  to  apply  them  in  a variety  of  situations.  School 
mathematics,  for  example,  provides  abstract  principles  for  different  kind  of  math  operations.  However,  as  many 
researchers  have  come  to  realize,  the  students  often  cannot  apply  the  rules  and  principles  in  real  life  situations.  Rather, 
they  tend  to  treat  them  as  something  to  be  memorized.  When  instructional  content  is  taught  context  independently, 
students  are  less  likely  to  be  able  to  see  the  utilities  of  information  sources  in  real  life  situations.  (CTGV  1992). 

Anderson  (1980,1982)  proposes  that  there  are  two  kinds  of  knowledge  representations,  declarative  knowledge  and 
procedure  knowledge.  Declarative  knowledge  consists  of  network  of  schema  and  procedural  knowledge  consists  of  network 
of  goal  oriented  production  rules  in  "If-Then"  format.  When  students  encounter  novel  situation,  they  rely  on  existing 
networks  to  make  inference  and  generate  solutions.  Learning  is  the  process  oi  connecting  new  information  to  the 
existing  schema  and  production  network  and  adjusting  the  existing  network  to  assimilate  the  new  information.  When 
instructional  content  domain  is  well  structured,  the  maps  of  schema  network  and  production  system  can  be  clearly 
defined.  Instructional  programs  based  on  systematic  instructional  design  models  can  be  very  effective  in  this  situation. 
However  in  many  cases  we  are  facing  a more  complex  world. 

The  complexity  of  situation  is  reflected  in  at  least  two  aspects. 

First,  learners  construct  their  meaning  and  their  knowledge  representation  according  to  their  physical, 
psychological  and  social  context  and  prior  experience  (Duffy  and  Jonassen,  1992).  Each  student  constructs  his/her 
understanding  and  knowledge  representations  in  learning  experience.  While  common  understanding  and  communication 
are  made  possible  by  social  negotiation  of  meaning,  learning  is  a process  of  active  seeking  of  personally  owned  meaning 
in  context. 

Second,  many  domains  of  learning  are  illstructured  (Spiro,  et  al.,  1987).  This  illstructuredness  is  reflected  by  the 
fact  that  there  is  no  apparent  set  of  concepts  or  principles  that  are  constantly  accurate  and  applicable  to  every  case  in  the 
domain.  Each  case  is  different  from  any  other  cases,  therefore,  predefined  production  rules  often  fail  to  address  certain 
situations  and  rigid  schema  structure  can  not  describe  complexity  of  the  domain.  Spiro  and  associates  (Spiro,  Coulson  et 
al.  1988)  propose  that  one  of  the  major  problems  of  traditional  instruction  is  reductive  bias.  Instruction  often  over-rely 
on  a single  or  limited  number  of  examples  or  prototypes,  therefore,  in  many  illstructured  domains,  complex  structures  arc 
artificially  oversimplified.  The  result  is  compartmentalized  knowledge  with  rigid  structures  that  are  not  very  useful  or 
usable  in  real  life  problem  solving  situations.  Memorization  often  takes  the  place  of  the  hoped-for  transfer  of  the  abstract 
knowledge. 

Cognitive  flexibility  theory  was  proposed  (Spiro,  et  al.,  1987,  Spiro,  et  al.,  1992)  to  help  students  to  master 
the  complexity  of  instructional  contents,  especially  in  advance  and  illstructured  domains.  Cognitive  flexibility  theory  is 
based  on  the  idea  that  learners  should  be  exposed  to  as  many  perspectives  of  the  knowledge  as  possible  and  they  should 
also  experience  different  cases  of  the  domain.  By  "Criss-crossing  the  domain  in  different  cases  and  from  different 
perspectives,  the  students  will  be  able  to  achieve  flexible  understanding  of  the  domain  and  its  complexity.  With  the 
flexible  understanding,  they  will  also  be  able  to  assemble  their  knowledge  to  solve  new  problems. 

To  address  the  question  ‘how  do  we  help  students  to  acquire  knowledge  and  skills  that  are  usable  in  real  life 
situation?”,  anchored  instruction  (CTGV,  1990;  CTGV,  1993),  as  a prime  example  of  case  based  situated  learning 
environment,  provides  an  authentic  and  generative  learning  environment  in  which  students  generate  subgoals  to  meet  the 
challenges  afforded  by  the  case.  Very  rich  context  information  helps  students  learn  and  use  mathematics  in  the  way  that 
they  can  see  how  information  can  be  relevant  and  useful  to  solving  the  problems,  rather  than  just  something  to 
memorize.  Brown,  Collins  and  Duguid  (1989)  argue  that  for  learners  to  be  able  to  obtain  usable  knowledge,  learning 
should  be  situated  in  authentic  context  and  activities.  Contextualization  of  learning,  or  learning  in  situ,  therefore,  is 
believed  to  be  an  effective  way  to  help  the  students  achieve  meaningful  learning.  Case  based  learning  environments  arc 
suggested  to  provide  rich  context  for  learning  and  meaning  making. 
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While  cognitive  flexibility  theory  does  not  particularly  prescribe  that  the  cases  for  the  students  to  criss-cross 
should  be  authentic  real  life  cases,  it  is  not  difficult  to  see  that  cognitive  flexibility  theory  can  go  hand  in  hand  with 
situated  learning  in  case-based  environments,  especially  when  the  domain  of  instruction  is  '.Hstructured  and  the  use  of 
domain  knowledge  is  often  expected  in  real  life  situations.  For  example,  almost  all  business  students  have  some  exposure 
of  management  information  system  (MIS).  One  of  the  most  important  concept  in  MIS  is  that  information  is  shared  in 
enterprises.  However,  for  different  people  in  the  organization,  an  information  system  provides  different  kinds  of  access 
and  different  information  systems  function  differently  in  different  business  settings.  Apparently,  this  is  not  a clear-cut 
topic.  The  better  the  students  understand  the  complexity,  the  more  likely  they  will  function  effectively  in  real  business 
operations.  What  if  we  design  a learning  environment  that  incorporates  the  assumptions  and  strategies  of  both  situated 
learning  and  cognitive  flexibility  theory  to  teach  enterprise  information  system? 

Purpose  of  the  Study 

The  purpose  of  this  study  was  to  compare  the  effectiveness  of  two  different  lesson  structures  on  helping  students 
make  inferences  and  predictions  after  studying  a computer-based  lesson  of  interdepartmental  information  system.  This 
subject  topic  was  required  by  undergraduate  business  majors  at  most  universities. 

There  were  two  different  implementations  based  on  different  lesson  structure,  which  were  case  based  cognitive 
flexibility  structure  (case-based  structure  henceforth)  and  concept  based  lesson  structure  with  examples  (concept-based 
structure  henceforth). 

Case-based  lesson  was  implemented  as  a cognitive  flexibility  hypertext.  There  were  four  cases  in  the  lesson. 

Two  cases  were  generic  cases  in  which  context  information  was  fairly  simple  and  the  other  two  were  very  rich  cases  with 
heavy  contextual  information  such  as  where  the  stories  happened,  what  kind  of  enterprises  they  were  and  so  on.  Different 
departments  of  the  enterprises  were  represented  by  different  on  screen  figures  of  the  departmental  heads  with  talking 
balloons.  Their  perspectives  were  presented  by  the  words  of  the  department  heads  with  the  cases  as  contextual  support. 
The  talking  balkons  contained  reflections  of  these  department  heads  regarding  to  how  information  was  accessible  to  the 
department,  how  the  department  contributed  to  the  shared  information  system,  and  how  the  information  system  helped 
and  influenced  their  business  plans  and  practices.  The  case  scenarios  provided  a context  on  which  they  could  reflect. 

The  implementation  of  concept-based  lesson  presented  the  basic  lesson  content  first.  The  basic  lesson  content 
covered  similar  information  provided  by  generic  cases  in  case-based  lesson  implementation.  However,  the  information 
was  presented  as  concepts  and  principles  in  outline  and  bullets  format.  After  the  basic  information  presentation,  two 
examples  were  used  as  support  of  the  concepts  and  principles.  The  examples  used  were  similar  to  the  context  rich  cases  in 
case-based  implementation,  however,  the  presentation  did  not  emphasize  the  contextual  information. 

The  study  tests  the  following  hypotheses  about  lesson  structure. 

1 . Students  in  case-based  lesson  make  better  inferences  from  given  information. 

2.  Students  in  case-based  lesson  predict  outcomes  of  new  situation  more  accurately  based  on  new  information. 

Method 

Participants 

Three  hundred  and  eighty-one  sophomore  business  students  from  a large  eastern  university  comprised  the 
sample.  They  did  not  have  prior  knowledge  about  the  subject  matter  area  covered  by  the  lessons  and  volunteered  to 
participate  this  study. 

Measurements 

At  the  end  of  the  lesson,  the  students  were  given  a test  with  10  multiple  choice  questions  on  prediction,  9 
multiple  choice  questions  on  inference.  These  questions  were  randomly  mixed.  The  time  that  students  spent  on  the 
program  was  recorded.  The  following  are  example  questions  for  prediction  and  inference  respectively. 

• Predication: 

The  Profit-Vision  program  that  analyzes  all  of  the  stores’  sales  should  produce  what  operational  effect: 

a.  quicker  collection  of  receivable 

b.  slower  collection  of  receivable 

C.  higher  in-store  inventories 

d.  lower  in-store  inventories 
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• Inference: 

The  most  important  advantages  of  Frito-Lay’s  information  system  are  centralizing  the  data  collection  and: 

a.  providing  regional  managers  access 

b.  centralizing  data  processing 

C.  expanding  the  number  of  collection  sites 

d.  predicting  the  volume  of  sales 

After  finishing  the  lessons,  students  also  took  the  first  part  of  Necessary  Arithmetic  Operations  Test  (French, 
Ekstrom  et  al.  1963),  which  measured  the  general  reasoning  factor.  This  test  consisted  of  15  multiple-choice  questions 
that  required  the  test  taker  to  determine  what  numerical  operations  were  required  to  solve  arithmetic  problems  without 
actually  having  to  carry  out  the  calculations. 

Treatments  and  Procedure 

Two  versions  of  the  lesson  were  developed.  The  first  version  was  a concept-based  implementation  which  used 
examples  to  help  on  conveying  main  concepts  and  principles  to  the  students.  This  version  of  lesson  was  used  in  concept- 
based  group.  The  second  version  was  case-based.  It  was  implemented  as  a cognitive  flexibility  hypertext  environment 
with  rich  cases  and  contextual  information.  These  two  versions  were  placed  on  the  college  LAN  server  under  different 
names.  Bach  student  copied  one  of  the  three  versions  onto  his/her  own  disk.  After  students*  finishing  the  lesson,  the 
responses  were  recorded  on  the  disks. 

The  student  volunteers  were  randomly  assigned  to  one  of  the  two  groups  using  one  of  these  different  versions. 
They  were  asked  to  work  independently.  The  students  finished  the  lesson  in  a period  of  one  week  and  turned  in  their 
answers  on  floppy  disks  so  that  their  answers  could  be  retrieved. 

Results 

There  was  no  significant  difference  between  the  average  amount  of  time  spent  by  the  two  groups  (Ps>.05). 

Table  1 presents  the  average  inference  and  predication  scores  in  case  based  treatment  and  concept  based  treatment 
respectively. 


Table  1,  Average  of  Inference  Scores  and  Predications  Scores  in  Two  Treatments 


Case  Based  Structure 

Concept  Based  Structure 

Inference 

4.11 

3.80 

Predication 

5.09 

5.01 

A regression  of  Predication  Scores  on  Time  Spent  on  Task  and  scores  of  Necessary  Arithmetic  Operations  Test 
(NAOT)  is  present  in  the  table  below: 


Table  2.  Regression  of  prediction  on  time  spent  on  task  and  NAOT 


Predictor 

Coefficient 

St.  Dev. 

t-ratio 

_E 

Constant 

2.78 

0.27 

10.29 

0.000 

Time 

2.98  E'4 

6.75  E 5 

4.40 

0.000 

NAOT 

0.20 

0.03 

7.27 

0.000 

s = 2.032  R-sq  = 18.1%  R-sq(adj)  = 17.7% 


A regression  of  Inference  Scores  on  Time  Spent  on  Task  and  scores  of  Necessary  Arithmetic  Operations  Test  is 
present  in  the  table  below: 


Table  3.  Regression  of  inference  on  time  s 

:>ent  on  task  and  NAOT 

Predictor 

Coefficient 

St.  Dev. 

t-ratio 

p 

Constant 

2.07 

0.23 

9.05 

0.000 

Time 

2.16  E 4 

5.71  E5 

3.79 

0.000 

NAOT 

0.18 

0.02 

7.45 

0.000 

s = 1 .721  R-sq  = 17.5%  R-sq(adj)  = 17.0% 
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Table  2 and  Table  3 show  that  both  time  and  NAOT  contribute  significantly  to  the  variances  in  distributions  of 
Predication  Scores  and  Inference  Scores. 

Using  scores  of  Necessary  Arithmetic  Operations  Test  (NAOT  scores)  as  covariate  for  Predication  and  Inference 
scores,  two  different  treatments  as  independent  variables,  Table  3 and  Table  4 present  the  results  of  Analysis  of 
Covariance: 


Table  4.  Analysis  of  covariance  for  prediction  with  NAOT  scores  as  covariate 


Source 

DF 

ADJ  SS 

MS 

F 

P 

Covariate 

(NAOT) 

1 

265.49 

265.49 

61.15 

0.000 

Treatment 

1 

0.19 

0.19 

0.04 

0.835 

Error 

378 

1641.13 

4.34 

Total 

380 

1907.15 

Table  5.  Analysis  of  covariance  for  inference  with  NAOT  scores  as  covariate 


Source 

DF 

ADJ  SS 

MS 

F 

P 

Covariate 

(NAOT) 

1 

192.62 

192.62 

63.07 

0.000 

Treatment 

1 

7.61 

7.61 

2.49 

0.115 

Error 

378 

1154.37 

3.05 

Total 

380 

1356.05 

Using  scores  of  Time  spent  on  task  as  covariate  for  Predication  and  Inference  scores,  two  different  treatments  as 
independent  variables,  Table  5 and  Table  6 present  the  results  of  Analysis  of  Covariance: 


Table  6.  Analysis  of  covariance  for  predication  with  time  as  covariate 


Source 

DF 

ADJ  SS 

MS 

F 

P 

Covariate 

(NAOT) 

1 

130.07 

130.07 

27.67 

0.000 

Treatment 

1 

3.11 

3.11 

0.66 

0.416 

Error 

378 

1776.55 

4.70 

Total 

380 

1907.15 

Table  7.  Analysis  of  covariance  for  inference  with  time  as  covariate 


Source* 

DF 

ADJ  SS 

MS 

F 

P 

Covariate 

(NAOT) 

1 

78.31 

78.31 

23.33 

0.000 

Treatment 

1 

14.46 

14.46 

4.31 

0.039 

Error 

378 

1268.68 

3.36 

Total 

380 

1356.05 

With  either  NAOT  or  Time  scores  as  covariate,  these  results  indicate  that  there  were  no  significant  different 
between  students’  scores  on  predication  in  the  two  treatments  (P<0.835  and  P<0.416  respectively). 

With  NAOT  scores  as  covariate,  ANCOVA  for  Inference  scores  did  not  yield  significant  results  (p<0.1 15). 
However,  it  is  worth  noticing  that  when  the  time  on  task  is  use  as  covariate,  ANCOVA  for  Inference  scores  yielded 
significant  results  (P<0.039).  This  indicates  that  if  we  partition  out  the  variance  brought  about  by  how  much  time  the 
students  spent  on  task,  the  different  treatments  then  demonstrate  significant  impacts  on  the  students’  capability  to  make 
inference.  The  students  in  case  based  treatment  performed  better  on  making  inference  than  those  in  concept  based 
treatment. 
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Discussion 

The  results  indicate  that  both  the  time  cn  task  and  NAOT  scores  contribute  positively  to  inference  and 
predication  scores.  The  more  time  the  students  spent  on  task,  the  higher  their  inference  and  predication  scores  tend  to  be. 
While  NAOT  is  a measure  of  flow  of  reasoning(French,  Ekstrom  et  al.  1963),  it  is  reasonable  to  believe  that  time  on 
task  is  an  indicator  of  how  much  mental  efforts  that  the  students  made  in  the  lessons.  It  is  therefore  not  surprising  at  all 
that  they  both  help  the  students  to  perform  better  in  make  predications  and  inferences. 

By  using  ANCOVA  as  a means  of  statistical  control,  we  were  able  to  isolate  the  effect  of  instructional 
treatments,  lesson  structures  in  this  case,  by  separating  the  contribution  of  time  and  NAOT  scores.  As  table  4 to  table  7 
indicate,  different  lesson  structures  did  not  contribute  to  the  variance  in  predication  score.  In  other  words,  the  difference  in 
the  students*  performance  of  making  predication  is  statistically  insignificant. 

However,  if  we  take  the  contribution  of  time  on  task  into  account  by  using  it  as  a covariate,  it  became  clear  that 
students  in  case-based  treatment  perform  better  in  making  inferences.  This  provided  some  empirical  data  that  support  both 
cognitive  flexibility  theory  (Spiro  et.  al.  1987,  1992)  and  situated  learning  theory  (Collins,  1991).  In  case-based  lesson, 
the  different  perspectives  served  as  different  themes  that  helped  the  students  criss-cross  the  knowledge  landscape  in 
different  cases.  These  flexible  knowledge  representations  helped  them  in  making  inferences  from  given  information.  The 
rich  contextual  information  also  helped  students  to  make  rich  connections  in  their  schema  and  production  network 
(Anderson,  1980).  These  relatively  more  complete  networks  also  helped  the  students  to  infer. 

Conclusions 

The  time  that  the  students  spent  on  tasks  and  the  reasoning  ability,  measured  by  Necessary  Arithmetic  Operation 
Test,  contributed  significantly  to  the  students*  performance  of  make  inferences  and  predications.  The  students  performed 
equally  well  in  making  predications  in  different  lesson  structures  (cased  based  and  concept  based).  In  making  inference 
from  given  information,  the  students  in  case  based  lesson  performed  better  than  those  in  concept  based  lesson  did.  It  is 
suggested  that  the  multiple  perspectives  and  rich  contextual  information  in  case  based  lesson  structure  afforded  students  to 
understand  the  inter-relationship  and  connect  the  information  into  better  representations,  which  helped  the  inference 
making  process. 
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Background 


Widespread  change  is  dramatically  altering  the  traditional  face  of  the  workplace.  Offermann  and  Gowing  (1990) 
state,  MThe  nature  of  work,  the  work  force,  and  the  workplace  have  undergone  and  will  continue  to  undergo  enormous 
change,  bringing  both  upheaval  and  opportunity  for  those  involved  in  organizations"  (p.  95).  This  rapid  change  is 
creating  new  problems  and  conflicts  which  call  for  new  interventions  and  strategies  (Dick,  1993;  Goldstein  and  Gilliam, 
1990). 


One  of  the  emerging  problems  in  the  field  of  Instructional  Development  (ID)  is  the  widening  gap  between  theory 
and  practice.  Consequently,  practitioners  are  faced  with  a dilemma.  On  the  one  hand,  they  would  like  to  apply  the  well- 
structured  theories  learned  from  schools.  On  the  other  hand,  they  find  that  these  precise  theories  are  less  practical  in  the 
field.  All  professionals  in  the  "bottom-line  driven"  corporate  world,  must  adopt  a more  flexible,  situational  approach  in 
order  to  get  the  job  done  as  smoothly  and  as  cost-effectively  as  possible.  Corporate  ID  practitioners  are  no  exception 
(Boutwell,  1979;  Dick,  1993;  Rossett,  1986;  Stolovitch,  1981). 

This  presentation  will  first  examine  the  gap  between  ID  theory  and  practice.  A conceptual  framework  will  then  be 
proposed  to  illuminate  how  expert  ID  practitioners  work  in  the  corporate  world. 

Gap  between  ID  Theory  and  Practice 

Over  the  years,  the  bulk  of  instructional  development  research  at  the  macro  level  has  focused  on  model-building 
(Andrews  & Goodson,  1980;  Taylor  & Doughty,  1988).  Theoreticians  identified  critical  ED  component  tasks,  organized 
them  in  a logical,  linear  sequence,  and  suggested  that  practitioners  follow  them  systematically  and  thoroughly  (Gustafson 
& Tillman,  1991).  But,  this  strictly  step-by-step  procedure  does  not  reflect  the  ID  process  as  practiced  in  the  field,  since 
it  is  what  works  that  counts  in  the  product-oriented  corporate  world. 

Until  recently,  little  formal  research  had  addressed  this  critical  issue  of  weighing  and  prioritizing  ID  tasks,  let 
alone  the  crucial  decision-making  processes  involved,  or  the  judgment  concerning  the  quality  of  decisions  made.  It  is 
also  noted  that  special  attention  is  now  being  given  to  social,  political  and  cultural  influences  in  corporate  education. 
Thus,  ID  professionals  may  have  to  immerse  themselves  in  diverse  human  and  social  factors  while  inquiring  into  this 
line  of  research  (Heinich,  1984;  McCombs,  1986;  Shrock,  1985;  Tessmer  & Wedman,  1990). 

Based  on  observations  of  recent  developments  in  both  ID  and  other  related  fields,  it  also  becomes  obvious  that  ID 
theory  and  practice  are  greatly  influenced  by  the  social  context  in  which  diverse  professions  interact  and  enrich  each  other. 
Thus  ID,  in  its  nature  and  practice,  must  be  considered  an  interdisciplinai7  profession.  By  taking  this  approach,  the 
existing  strengths,  experiences,  and  resources  of  the  ID  profession  can  be  optimally  enhanced. 

All  this  is  leading  to  the  emergence  of  a more  flexible,  situational  and  social  approach  to  ID.  This  context- 
sensitive  approach,  as  opposed  to  the  classical,  mechanistic  mode,  will  accelerate  the  development  of  a- coherent  set  of  ID 
theories  and  practices.  The  implication  which  emerges  from  an  examination  of  these  issues  in  the  literature  and 
professional  practice  in  the  field  is  the  urgent  need  to  narrow  the  gap  between  ID  theory  and  practice. 

Professional  Practice  in  Corporate  Education 

On  the  basis  of  the  authors'  lengthy  experience  in  the  field  of  corporate  consulting,  we  propose  a conceptual 
framework  to  describe  how  expert  ID  practitioners  work  in  the  corporate  reality.  This  framework  consists  of  five  major 
themes,  which  are: 

(1)  cultural  context, 

(2)  situation  analysis  and  synthesis, 

(3)  client  and  other  stakeholders, 

(4)  performance  capacities  (capacity  analysis  and  professional  ethics), 

(5)  competitive  advantages  (business  strategy,  professional  conduct,  and  changing  roles). 
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As  Figure  1 illustrates,  under  the  umbrella  of  cultural  context,  a project  brings  the  practitioner  and  the  client 
systems  together.  The  practitioner  strives  for  the  project's  success  by  conducting  situation  analysis  and  synthesis, 
analyzing  performance  capacities,  and  cooperating  with  the  client  and  other  project  stakeholders.  The  practitioner  operates 
within  a framework  of  professional  ethics,  and  exercises  three  competitive  advantages  consisting  of  business  strategy, 
professional  conduct,  and  changing  roles. 

In  the  following  sections,  although  each  key  theme  is  addressed  separately,  it  should  be  noted  that  each  theme  has 
a unique,  sophisticated,  interwoven  relationship  with  the  others. 
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Figure  1:  Themes  of  Corporate  Education  (from  research  on  "The  impact  of  situational  factors  on 
the  corporate  instructional  development  practitioner's  decision  making,"  Liang,  C.,  1994) 


Cultural  Context 

In  this  paper,  cultural  context  is  defined  as  the  specific  context  in  which  the  ID  project  is  carried  out  under  a set  of 
traditions,  values,  beliefs,  and  influences  of  the  contemporary  society.  In  other  words,  the  cultural  context  of  a specific 
society  is  closely  associated  with  the  shared  values  and  traditions  in  that  community.  It  engenders  a complicated  network 
of  that  group's  concerns  and  beliefs  in  which  the  conventional  traditions  are  intertwined  with  emerging  contemporary 
issues.  The  new  values  resulting  from  contemporary  concerns  may  conflict  with  the  tradition  ones. 

This  cultural  context  acts  as  a filter  to  help  ID  practitioners  preserve  those  things  which  fit,  but  to  leach  out  those 
which  do  not.  It  directs  their  conduct:  it  leads  them  to  think,  to  believe  and  to  act. 

Situation  Analysis  and  Synthesis 

The  real-world  situations  which  corporate  ID  practitioners  faced  are  dynamic  and  few  are  covered  by  the  theoretical 
methods  learned  from  textbooks.  From  the  outset  of  each  project,  in  fact  even  before  they  take  on  a project,  practitioners 
need  to  closely  analyze  the  situations  they  are  likely  to  encounter,  and  to  constantly  synthesize  their  findings. 

In  other  words,  in  this  dynamic  environment,  expert  practitioners  change  or  modify  their  approaches  as  the  ID 
process  evolves.  As  each  situation  arises,  they  interpret  its  meaning  by  combining  the  new  contextual  information  with 
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their  previous  information  and  experiences.  Their  subsequent  interpretations  bring  into  focus  the  risks  and  opportunities 
inherent  in  this  new  situation.  The  practitioners  then  take  action  to  minimize  these  risks  and  optimize  the  opportunities. 

Experienced  practitioners  are  always  on  the  alert  for  unexpected  or  rapid  changes.  They  continually  probe  for 
relevant  information,  asking  themselves  if  there  are  any  better  options.  They  flexibly  adjust  their  mindsets  and  take 
action,  first  by  integrating  the  new  inputs,  and  then  by  visualizing  the  consequences  of  their  subsequent  choices.  This 
ongoing  process  allows  them  to  take  account  of  their  emerging  insights  and  to  revise  their  decisions  accordingly. 

For  the  most  part,  ID  decisions  in  professional  practice  cannot  be  judged  simply  as  right  or  wrong.  Much  depends 
on  the  practitioners'  interpretations  of  the  situations  and  on  the  possible  outcomes  they  wish  to  achieve.  Thus,  situation 
analysis  and  synthesis  is  aimed  at  achieving  full  utility  of  the  resources  and  the  interventions  available  at  that  specific 
point  in  time.  How  well  practitioners  optimize  this  integration  largely  depends  on  how  well  they  understand  the  problem 
and  the  client  system. 

Client  and  Other  Stakeholders 

To  meet  their  client’s  needs  and  to  carry  out  their  projects  effectively,  experienced  practitioners  direct  their  energies 
to  the  right  place  by  asking,  "Who  is  the  real  client?"  and  by  strategically  distinguishing  between  "their  success"  and 
"their  clients'  success." 

The  aim  of  expert  practitioners  is  not  only  to  satisfy,  but  to  delight  their  client  and  to  impact  the  client's 
organization.  T i.i  this  end,  they  collect  information  which  enable  them  to  grasp  the  client's  intention,  agenda,  policy, 
expectations,  and  standard.  Not  only  do  they  comprehend  their  client's  needs,  but  also  those  of  the  other  project 
stakeholders.  Expert  corporate  practitioners  are  committed  to  having  a deep  understanding  of  both  their  clients'  industries 
and  social  dynamics  which  smooth  out  the  process  and  facilitate  their  projects'  successes. 

Expert  practitioners  also  create  communication  channels  to  listen  to  those  clients  who  are  closest  to  the  problems 
and  their  solutions.  They  are  sensitive  to  and  follow  their  clients'  organizational  ecology  to  increase  their  clients'  buy-in. 
These  practitioners  also  play  different  roles  in  response  to  their  clients'  various  needs  and  to  the  situations  they  encounter. 

When  conflicts  arise  among  their  clients  and  other  project  stakeholders,  even  at  times  involving  the  practitioners 
themselves,  the  practitioners  may  be  caught  in  an  ethical  dilemma.  This  growing  awareness  of  and  concern  for  the 
stakeholders'  needs  and  benefits  demonstrates  a new  movement  beyond  "customer  satisfaction"  to  "social  satisfaction." 

Performance  Capacities 

The  practitioners'  performance  capacities  can  be  divided  into  two  categories:  capacity  analysis  and  professional 
ethics.  On  the  one  hand,  capacity  analysis  includes  an  examination  of  the  professional  competencies,  the  resources  and 
other  tangible  means  necessary  for  the  practitioners  to  carry  out  their  projects.  Professional  ethics,  on  the  other  hand, 
represents  the  impact  of  spiritual  power  such  as  values,  mindset  and  morals  on  the  practitioners’  decisions  and  actions. 
Unless  the  practitioners  dovetail  their  performance  capacities  with  their  clients'  needs,  their  projects  are  unlikely  to  have 
fruitful  outcomes. 

Capacity  Analysis 

In  this  section,  capacity  analysis  is  discussed  at  three  levels:  organization,  team,  and  individual.  It  is  capacity 
analysis  at  these  three  levels  which  allows  each  practitioner  to  predict  the  most  productive  route  to  take  during  a project, 
and  to  orchestrate  those  performances  which  will  produce  a coherent  whole. 

Organization  capacities 

Capacity  analysis  at  the  organizational  level  includes  an  examination  of  the  history,  mission,  values,  culture, 

leadership,  resources,  structure  and  politics  of  the  practitioner’s  own  organization. 
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Team  capacities 


At  the  team  level,  an  accurate  capacity  analysis  helps  the  practitioners  to  define  and  bring  together  the  appropriate 
capabilities  and  resources  which  enable  them,  as  project  managers,  to  lead  their  teams'  efforts  toward  their  project 
missions. 

Individual  capacities 

Capacity  analysis  at  the  individual  level  includes  a close  inquiry  by  the  practitioners  into  each  potential  team 
member's  competencies  and  the  resources  at  hand.  This  allows  them  to  marshal  and  allocate  the  appropriate 
capacities  and  to  make  best  use  of  them  at  the  individual  level. 

In  sum,  the  organization,  team  and  individual  capacities  available  to  the  practitioners  must  match  their  clients' 
needs  in  order  to  guarantee  a win-win  result. 

Professional  Ethics 

The  second  category  of  the  practitioners’  performance  capacities  is  professional  ethics.  Professional  ethics 
represents  the  impact  of  spiritual  power  such  as  values,  mindset  and  morals  on  the  practitioners'  operations.  It  is  the 
criteria  to  which  the  practitioners  refer  when  they  wish  to  justify  a decision.  Professional  ethics  are  objectively  regulated 
by  system  rules  such  as  laws,  formal  guidelines  and  societal  norms,  and  are  subjectively  bounded  by  personal  values, 
beliefs  and  cultural  background. 

Expert  practitioners  consistently  demonstrate  high  ethical  standards  which  guide  their  personal  and  professional 
conduct.  Cultural  context  and  the  diverse  conflicts  of  interest  which  arise  among  the  client  and  various  project 
stakeholders  have  a determining  influence  on  the  practitioners'  ethical  practices.  Practitioners  constantly  mediate  the 
tensions  between  a generalized  duty  owed  to  society  and  to  their  profession  vs.  the  specific  duty  owed  to  a particular 
client.  And,  they  reconcile  the  tensions  between  the  ideal  of  a general  principle  and  the  reality  of  a particular  practice. 

Competitive  Advantages 

Expert  corporate  ID  practitioners  strive  for  their  projects'  success  by  exercising  three  competitive  advantages: 
business  strategy,  professional  conduct,  and  changing  roles.  It  is  these  three  competitive  advantages  which  make  these 
practitioners  uniquely  successful. 

First,  business  strategy  represents  the  collective  schema  of  the  practitioner’s  organization,  which  creates  a vision 
and  sets  up  a direction  for  future  action.  Next,  professional  conduct  signifies  the  actions  which  mobilize  all  the  resources 
necessary  for  the  practitioner  to  enact  the  constructed  business  strategy.  Finally,  changing  roles  symbolize  both  the 
capacity  for  flexibility  within  a project  and  a movement  toward  continual  professional  development  as  the  practitioner 
works  in  the  field  and  grows  from  the  experience. 

Business  Strategy 


Business  strategy  is  the  unifying  idea  that  links  together  the  functional  areas  in  a company  and  relates  its  activities 
to  its  external  environment.  It  results  in  no  immediate  productive  action,  but  sets  a general  direction  in  which  the 
company’s  position  will  grow  and  develop.  It  is  the  means  to  the  company’s  ends. 

In  implementing  business  strategy,  expert  practitioners  go  beyond  success.  Not  only  do  they  strive  to  satisfy 
their  clients'  needs,  they  aim  to  delight  their  clients.  They  push  the  limits  of  their  professional  boundaries,  and  promote 
their  own  organization’s  image  in  society.  At  the  same  time,  they  extend  the  focus  of  their  concerns  beyond  their  clients' 
needs  to  encompass  the  other  relevant  project  stakeholders'  benefits.  This  signifies  a movement  from  "customer 
satisfaction"  toward  "social  satisfaction". 
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Professional  Conduct 


Professional  conduct  is  the  second  of  the  expert  practitioners’  three  competitive  advantages.  While  business 
strategy  sets  the  general  direction  for  development,  professional  conduct  represents  the  practitioners’  actions  which  enable 
the  implementation  of  the  constructed  business  strategy. 

The  professional  conduct  of  expert  practitioners  is  directly  influenced  by  each  theme  discussed  in  this  paper,  or  by 
various  combinations  of  these.  In  striving  for  their  projects’  success,  the  practitioners’  professional  conduct  is  always 
aimed  at  achieving  a win-win  outcome  for  their  clients,  the  relevant  stakeholders,  their  own  organizations  and 
themselves. 

Changing  Roles 

Changing  roles  is  the  third  competitive  advantage  which  expert  practitioners  exercise.  Throughout  each  project, 
they  wear  "multiple  hats"  (they  exhibit  the  ability  to  function  in  multiple  capacities)  to  meet  the  wide  range  of  their 
clients’  needs.  At  the  same  time,  they  also  apply  the  experience  they  accumulate  from  their  projects  to  facilitate  their 
own  professional  growth.  In  other  words,  changing  roles  represents  the  process  of  continual  professional  development 
from  a single-specialty  practitioner  to  a holistic  organizational  consultant. 

But,  no  matter  which  roles  these  practitioners  play  throughout  the  course  of  their  projects  and  their  professional 
lives,  they  always  play  the  role  of  competent  helper. 


Core  Bibliography 

Andrews,  D.H.  and  Goodson,  L.A.  (1980).  A comparative  analysis  of  models  of  instructional  design.  Journal  of 
Instructional  Development.  2(4),  2-16. 

Boutwell,  R.C.  (1979).  Instructional  systems  development  in  the  next  decade.  Journal  of  instructional  Development. 

2(3),  31-35. 

Dick,  W.  (1993).  Enhanced  ISD:  A response  to  changing  environments  for  learning  and  performance.  Educational 
Technology  ,22(2),  12-16. 

Goldstein,  I.L.  and  Gilliam,  P.  (1990).  Training  system  issues  in  the  year  2000.  American  Psychologists,  45(2),  1 34- 
143. 

Gustafson,  K.L.  and  Tillman,  M.H.  (1991).  Introduction.  In  L.J.  Briggs,  K.L.  Gustafson,  and  M.H.  Tillman  (Eds.), 
Instructional  design:  Principles  and  applications.  (2nd  Ed.).  Chapter  l.  Englewood  Cliffs,  NJ:  Educational 
Technology. 

Heinich,  R.  (1984).  The  proper  study  of  instructional  technology.  Educational  Communication  and  Technology  Journal 
(ECTJ).  22(2),  6784. 

Liang,  C.  (1994).  The  impact  of  situational  factors  on  the  corporate  instructional  development  practitioner’s  decision 
making.  An  un-published  dissertation  of  Indiana  University. 

McCombs,  B.L.  (1986).  The  instructional  systems  development  (ISD)  model:  A review  of  those  factors  critical  to  its 
successful  implementation.  Educational  Communications  and  Technology  Journal  (ECTJ).  24(2),  67-81. 

Offermann,  L.R.  and  Gowing,  M.K.  (1990).  Organizations  of  the  future:  Changes  and  challenges.  American 
Psychologists.  45(2).  95108. 

Rossett,  A.  (1986).  What  your  professor  never  told  you  about  the  mundance  practice  of  instructional  design.  Paper 
presented  at  the  Annual  Convention  of  the  Association  for  Educational  Communications  and  Technology,  Las 
Vegas,  NV,  1986.  ED267790. 

Shrock,  S.A.  (1985).  Faculty  perceptions  of  instructional  development  and  the  success/failure  of  an  instructional 

development  program:  A naturalistic  study.  Educational  Communications  and  Technology  Journal  (ECTJ).  22(0. 
16-25. 


O 

ERIC 


BEST  COPY  AVAILABLE 


Stolovitch,  H.D.  (1981).  Preparing  the  industrial  and  educational  instructional  developer:  Is  there  a difference?  N$PI 
Journal.  20(1),  29-30. 

Taylor,  R.  and  Doughty,  P.L.  (1988).  Instructional  development  models:  Analysis  at  the  task  and  subtask  levels.  Journal 
of  Instructional  Development.  1 1(4).  19-28. 

Tessmer,  M.  and  Wedman,  J.F.  (1990).  A Layer-of-Necessity  instructional  development  model.  Educational  Technology 


),  28(2),  77-85. 


436 


Title: 


Using  a Hypertext  Environment  for  Teaching  Process  Writing:  An 
Evaluation  Study  of  Three  Student  Groups 


Authors: 


Linda  Lohr 

Horizon  Interactive,  Inc. 
and 


Steven  M.  Ross 
University  of  Memphis 
and 


Gary  R.  Morrison 
University  of  Memphis 


Abstract 

The  present  research  consisted  of  a comprehensive  evaluation  of  a hypertext  model  for  teaching  process  writing  at  the 
junior  high  and  high  school  level.  Interests  were  to  determine  how  two  teachers  and  three  different  age  groups  of  students 
used  and  reacted  to  the  model,  specifically,  its  embedded  design  features  of  model  stories,  note  cards,  idea  buttons,  mini- 
lessons, branching  buttons,  and  cut-and-paste-tools.  Results  showed  applications  of  the  embedded  features  to  vary  based 
on  teacher  attitudes,  feature  attributes  (e.g.,  ease  of  use  and  appeal),  and  student  characteristics.  Older  students  made  more 
usage  of  many  of  the  features,  but  were  less  positive  about  the  hypertext  model  given  their  greater  involvement  with 
completing  writing  assignments  rather  than  with  exploring  new  forms  of  writing.  The  implications  of  the  results  are 
discussed  regarding  the  instructional  design  and  classroom  implementation  of  new  technologies  for  teaching  process 
writing  strategies. 


Using  a Hypertext  Environment  for  Teaching  Process  Writing: 

An  Evaluation  Study  of  Three  Student  Groups 

Several  reports  have  recently  documented  the  decline  of  writing  skills  of  American  students  (National 
Assessment  of  Educational  Progress,  1990;  Walton,  1990).  Although  basic  composition  skills  appear  “adequate,” 
students  show  limited  success  with  writing  tasks  that  involve  higher-order  thinking  and  reasoning  (Applebee,  Langer,  & 
Mullis,  1990).  Directly  relevant  to  the  focus  of  the  present  research  is  the  difficulty  today’s  students  experience  in  using 
imaginative  writing  skills  to  create  original  stories  (Stein,  1986;  Walton,  1990). 

One  approach  to  the  challenge  of  improving  students’  higher-order  writing  ability  is  “process”  orientations  to 
teaching  writing.  In  contrast  to  “product”  orientations  that  emphasize  mechanical  skills  such  as  punctuation,  spelling, 
and  grammar,  process  orientations  stress  writers’  personal  construction  of  meaning  and  structure  (Resnick,  1987).  The 
process  orientation  also  emphasized  a social  context  of  student  conferencing  during  the  various  non-linear  writing  stages 
of  planning,  generating,  revising,  and  evaluating  (Bereiter  & Scardamalia,  1986;  McGee  & Richgels,  1990). 

The  following  four  assumptions  underlie  the  design  of  process  writing  strategies:  (a)  writing  is  a process  of 
constructive  problem  solving,  and  through  writing  practice,  students  develop  thinking  and  writing  skills;  (b)  experts  and 
novices  approach  the  process  differently;  (c)  writing  is  recursive,  drawing  on  many  stages  of  writing  randomly  rather  than 
linearly  or  sequentially;  and  (d)  the  strategy  the  writer  takes  is  dependent  on  the  purpose  and  nature  of  the  writing  task 
(Hildyard,  1992).  Most  school-based  process  models  of  instruction  describe  the  following  five  stages  (Calkins,  1986; 
Hillerich,  1985;  Madden,  Wasik,  & Petra,  1989;  McGee  & Richgels,  1990):  (a)  rehearsing  or  planning,  (b)  drafting,  (c) 
revising,  (d)  editing,  and  (e)  publication.  However,  a well  formulated  theory  does  not  guarantee  a successful  application. 
The  effectiveness  of  such  approaches  in  improving  students’  writing  skills  is  yet  to  be  established.  Probably  the  most 
widely  stated  criticism  is  that  process  approaches  are  applied  too  superficially  (Applebee  et  al„  1990;  Calkins,  1985; 
Resnick,  1987;  Stein,  1986).  Frequently  cited  consequence  is  that  students  may  not  be  able  to  transfer  process  skills 
instruction  to  the  types  of  writing  tasks  they  confront  in  real-life  contexts. 

The  problems  encountered  with  applying  theoretical  models  to  everyday  situations  are  not  unique  to  writing 
instruction.  Incorporating  thinking  skills  within  specific  disciplines  remains  an  important  challenge  for  educators  in  the 
domains  of  problem-solving  (Resnick,  1987)  and  addresses  the  need  for  the  development  of  prescriptive  research 
methodologies  (Clark,  1989). 

Computers  and  Process  Writing 

Word-processing  technology  is  generally  believed  to  support  process  orientations  because  it  carries  the  potential 
to  remove  many  of  the  mundane  and  time-consuming  barriers  to  writing  (Bruce,  1991;  Daiute,  1992).  A typical  claim  is 
that  students  who  spend  less  time  in  mechanical  activities  of  rewriting,  revising,  and  restructuring  can  spend  more  time 
problem  solving,  thinking,  and  planning.  Although  there  is  a widespread  belief  that  word  processing  facilitates  writing, 
research  has  yet  to  show  that  it  directly  fosters  writing  improvement  (Cochran-Smith,  1991 ). 

An  increasingly  used  computer  application,  hypertext,  is  intuitively  promising  because  its  non-linear  structure 
suggests  an  ideal  environment  for  writing  instruction  within  a process  framework.  Hypertext  is  the  label  for  computer- 
driven  displays  of  information  that  can  display  information  in  various  combinations.  In  a hypertext  environment, 
information  can  be  connected  or  linked  to  any  other  information  contained  in  that  environment.  Because  of  linking  and 
branching  capabilities,  the  structure  of  hypertext  is  frequently  compared  to  the  structure  of  human  memory  v ' ’onklin. 
1987;  Jonassen,  1991;  Warren,  1989),  and  as  such  can  be  designed  for  a multitude  of  instructional  uses. 


O 

ERIC 


J5* 


Features  of  hypertext  that  may  enhance  traditional  word-processing  capabilities  include  the  capability  to  revise 
and  restructure;  to  branch  to  alternative  arguments  and/or  descriptions  (Bolter,  1991);  to  embed  notes,  tips,  and 
elaborations;  to  store  writing  sequences  from  past  sessions  (Neuwirth,  Kaufer,  Chimera,  & Gillespie,  1987);  and  to 
model  expert  decisions  (Jonassen,  1991).  All  of  these  features  may  facilitate  non-linear  aspects  of  writing  and  suggest 
that  hypertext  may  be  an  attractive  tool  for  supporting  the  decisions  of  writers  while  they  are  structuring  knowledge. 

Although  hypertext  promises  great  potential  for  writing  instruction,  its  effectiveness  in  this  area  still  remains  in 
question.  Several  development  projects  illustrate  varied  applications  of  hypertext  writing  environments,  but  have  not 
encompassed  systematic  investigations  of  learner  processes  and  outcomes.  These  projects  include  (a)  Notec ards.  a 
collaborative  writing  project  (Trigg  & Suchman,  1989);  (b)  the  Writing  Environment,  featuring  planning,  writing,  and 
editing  modes  based  on  cognitive  learning  principles  (Smith  & Lansman,  1988);  (c)  Hvperstories.  a combination  of 
HyperCard™  and  a videodisk  program  used  for  teaching  and  story  writing  on  ecological  issues  (McLellan,  in  press);  (d) 
Multimedia  Stories,  a multimedia  composing  tool  (Daiute,  1992);  (e)  and  the  Apple  Classrooms  of  Tomorrow  study  of 
children  using  multimedia  software,  StoryShow.  in  co-authoring  stories  (Reilly,  1992). 

Puipose  of  the  Present  Research 

Problems  with  implementing  general  design  models,  such  as  process  writing,  are  widely  reported  in  many 
academic  disciplines  (Resnick,  1987)  and  have  become  a challenging  issue  for  the  field  of  instructional  design  and 
technology.  Hypertext  may  be  a promising  tool  for  addressing  the  problems  related  to  implementing  the  strategies  of 
processing  writing  within  a classroom  structure.  One  advantage  would  be  incorporating,  within  the  hypertext  design, 
strategies  that  handle  many  of  the  sizable  management  tasks  associated  with  the  process  approach.  Additionally,  the 
modeling  of  expert  knowledge  structures  important  to  story  writing  suggests  an  effective  way  to  teach  specific  thinking 
skills. 

The  present  study  investigated  the  effectiveness  of  a hypertext  writing  environment  specifically  designed  to 
facilitate  process  model  instruction.  The  program  contained  several  embedded  features  to  stimulate  and  support  usage  of 
task-relevant  problem  solving  and  thinking  skills  during  writing  activity.  Included  were  note-taking,  mini-lessons, 
teacher-  and  student-generated  writing  suggestions,  sample  stories,  branching  cards,  and  teacher-  and  student-modeled 
writing  samples.  The  basic  writing  model  consisted  of  six- week  writing  unit  in  which  students  were  trained  on  using 
hypertext,  read  sample  stories,  and  wrote  original  stories  using  the  embedded  features.  Research  outcomes  of  interest  were 
students’  usage  of  the  embedded  features,  students’  and  teachers’  experiences  during  the  implementation  period,  and  the 
nature  and  quality  of  process  writing  products.  The  research  orientation  represented  an  applied  descriptive  study  or  an 
evaluation  study  (Ross  & Morrison,  1994).  Despite  its  emphasis  on  a particular  instructional  model,  findings  were  also 
expected  to  suggest  general  principles  for  improving  the  effectiveness  of  similar  applications  (i.e.,  hypertext  contexts)  for 
writing  instruction  and  other  curriculum  areas.  Specific  research  questions  were  as  follows: 

1.  To  what  extent  do  the  embedded  features  of  hypertext  facilitate  process  model  instruction  in  the  categories  of: 

(a)  problem-solving  and  writing,  (b)  expert  modeling,  (c)  non-linear/recursive  writing,  and  (d)  story  writing 

strategies? 

2.  Is  usage  of  the  above  features  associated  with  improved  writing  products? 

3.  Are  there  age  and  gender  differences  in  embedded  feature  use  and  attitudes  toward  the  process  writing  model? 

Method 


We  employed  a descriptive,  naturalistic  evaluation  design  in  which  a hypertext  process  writing  model  was 
implemented  with  16  junior  high  and  22  high  school  students  over  an  eight-week  period.  The  model  was  implemented  as 
part  of  the  students’  regular  curriculum  and  classroom  activities.  Students  attended  a private  school  with  strong  liberal 
arts  orientation  and  that  serves  a mostly  upper-middle  class  student  population.  As  a function  of  the  naturalistic  design, 
special  characteristics  of  the  student  population,  and  relatively  small  sample  sizes,  multiple  data  sources,  both 
quantitative  and  some  qualitative,  were  used  in  conjunction  with  intensive  study  of  student  activities  during  all  phases  of 
the  study.  For  the  qualitative  elements,  we  followed  suggestions  in  the  literature  for  achieving  high  internal  validity  by 
incorporating  prolonged  engagement  (an  cight-wcf  k study),  persistent  observations,  triangulation  of  data  sources,  peer 
debriefing  (discussion  with  the  observer  about  motives,  rationales,  possible  biases,  etc.),  and  member  checking 
(confirmation  of  results  by  participants)  (Lincoln  & Guba,  1985;  also  see  Gallo  & Horton,  in  press). 
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The  junior  high  group  consisted  of  a 7th-grade  microcomputer  class,  taught  by  the  computer  teacher,  that  met 
for  four  50-minute  sessions  a week.  Eight  males  and  eight  females  were  in  the  class.  The  high  school  group  was  a 
combined  9th-  to  12th-grade  class  that  also  met  for  four  50-minute  weekly  sessions.  There  were  13  males  and  9 females 
in  that  class.  All  students  participated  in  the  researched  writing  tasks  as  part  of  their  normal  class  activities  which 
included  computer-based  writing  as  a component.  Ninth  graders  (n  = 10)  were  enrolled  in  a microcomputer  class 
emphasizing  writing,  while  the  1 0th- 1 2th  graders  were  enrolled  in  a “Writer’s  Workshop”  course  also  focusing  on 
writing.  (For  clarity,  we  will  henceforth  refer  to  this  group  as  the  WW  students.)  The  ninth-grade  and  WW  classes  were 
taught  by  the  same  high  school  teacher  who  specialized  in  English  and  computer-based  writing. 

According  to  the  high  school  teacher,  most  of  her  students  were  familiar  with  the  process  model  of  writing  as  a 
function  of  its  being  practiced  informally  by  most  language  arts  teachers  in  the  school.  The  microcomputer  teacher 
described  her  students  as  being  less  familiar  with  the  process  approach,  but  as  having  basic  computer  skills  (e.g.,  using  a 
mouse,  opening  and  saving  files,  etc.).  For  purposes  of  examining  how  students  differing  in  age,  experience,  and  course 
orientation  would  react  to  and  use  the  hypertext  writing  model,  we  felt  that  the  most  appropriate  breakdown  for 
comparison  would  consist  of  three  groups:  7th-graders,  9th-graders,  and  WW  students. 

Instructional  and  Evaluation  Materials 

The  hypertext  writing  program  was  implemented  over  an  eight-week  period,  of  which  two  weeks  were  used  to 
field  test  the  program  using  individual  students  and  small  groups  of  students,  and  six  weeks  were  used  for  classroom  story 
writing.  The  six-week  session  included  one  week  for  an  introduction  to  writing  in  hypertext  with  a practice  activity,  and 
five  weeks  of  story  writing  in  which  students  wrote  two  original  stories. 

The  Hypertext  Story  Writing  Environment 

Students  were  given  a HyperCard  stack  that  contained  five  embedded  strategies  (accessed  through  HyperCard 
buttons)  to  facilitate  usage  of  elements  of  the  process  model.  Figure  1 illustrates  the  incorporation  of  the  strategies;  a 
brief  discussion  of  each  strategy  follows  (for  more  detailed  descriptions,  see  Lohr,  1993). 


Figure  1 . A view  of  the  hypertext  writing  page  with  embedded  features. 

1.  Sample  storv.  The  Frog  Princess,  a Russian  folk-tale,  was  adapted  to  provide  a hypertext  sample  story 
incorporating  vhe  author’s  embedded  notes  and  examples  of  computer  branching  to  different  endings.  Students  were  given 
the  option  to  revise  the  Frog  Princess  by  creating  branches  to  different  beginnings,  character  descriptions,  and  character 
actions.  The  sample  story  was  intended  to  reinforce  a process  writing  orientation  by  conveying  writing  as  constructive 
problem-solving  in  which  the  author  reflects  on  and  tries  out  different  ideas. 

2.  Note  card  buttons.  Two  types  of  note  cards  were  linked  to  each  page  using  HyperCard  buttons  (see  Figure 
l ).  Goal-statement  note  cards  provided  an  edit  field  to  list  one’s  writing  goals.  Prompts  were  displayed  to  remind 
students  to  use  the  goal  statement  button  for  recording  their  writing  plans  before  leaving  the  page.  Editor  note  cards 
provided  edit  fields  for  the  teacher  (or  classmates)  to  record  feedback  comments  for  that  page.  These  fields  were  hidden 


from  view  until  the  critic  or  author  clicked  the  appropriate  button  to  view  the  fields.  Students  were  initially  told  and 
intermittently  reminded  to  read  an  embedded  mini-lesson  (see  below)  that  instructed  them  on  questions  to  ask  when 
reading  another  student’s  story.  The  goal  statement  and  editor  note  cards  were  used  primarily  to  support  four  process 
writing  components:  (a)  problem-solving  and  (b)  planning  through  the  setting  of  writing  goals  and  recording  of  writing 
problems,  (c)  expert  level  suggestions  through  the  editor  feedback,  and  (d)  revising. 

3.  Mini-lessons.  Six  mini-lessons  were  embedded  in  the  students’  writing  stack  following  the  suggestion  by 
Calkins  (1985)  that  brief,  informally-introduced  and  well-timed  instructional  units  covering  topics  important  to  writing 
development  should  be  made  available  as  needed  during  writing  activity.  Students  could  directly  access  any  of  the 
following  mini-lessons  during  their  writing:  (a)  basic  story  structure;  (b)  effective  branching  techniques;  (c)  choosing  a 
topic;  (d)  conferencing;  (e)  rehearsal  and  revision  strategies;  and  (f)  “to  do  today,’’  a list  of  teacher  suggestions  and 
reminders  by  clicking  on  the  Mini-lessons  button  (see  Figure  1).  The  stack  would  then  branch  to  the  appropriate  mini- 
lesson. The  mini-lessons  presented  instruction  on  two  major  process  writing  characteristics,  writing  as  a non-linear 
activity  and  usage  of  specific  story  writing  strategies. 

4.  Branching  buttons.  Branching  buttons,  provided  as  an  option  on  each  HyperCard  page,  allowing  the  student 
to  create  up  to  three  written  detours  from  each  writing  page  (see  right  page  of  Figure  1).  Branching  could  be  used  to  tell 
a story  from  different  perspectives  (e.g.,  a happy  or  sad  ending)  or  for  elaborating  and  exploring  story  ideas  for  possible 
use  in  the  actual  story.  The  branching  component  most  directly  addressed  the  non-linear  nature  of  process  writing,  along 
with  problem  solving,  and  planning.  After  clicking  a branching  button,  the  student  could  write  a new  section  which  was 
linked  to  the  original  story.  Each  branch  could  subsequently  be  evaluated  by  clicking  the  appropriate  branching  button 
which  would  then  display  the  branch. 

5.  Cut-and-paste  buttons.  Cut-and-paste  buttons  (see  Figure  1)  allowed  students  to  move  sections  of  text 
without  the  need  for  retyping.  This  feature  directly  supported  the  process  writing  components  of  revising  and  editing. 
Students  could  highlight  the  text  they  wanted  to  copy  or  paste  and  the  click  the  appropriate  button.  The  cut-and-paste 
buttons  were  added  as  an  alternative  to  the  Cut-and-Paste  items  under  the  Edit  menu. 

Data  Collection  Instruments 

Eight  types  of  instruments  were  employed  to  collect  quantitative  and  qualitative  data  relevant  to  the  research 
questions.  An  overview  of  the  instruments  is  provided  in  Table  1 ; a description  of  each  follows. 

Observer  notes.  Observer  notes  were  recorded  daily  by  the  first  author  during  study  writing  sessions.  These 
notes  contained  descriptions  and  impressions  of  the  activities  as  well  as  reactions  expressed  by  students  and  teachers 
regarding  their  experiences  with  the  writing  tasks. 

Frequency  counts  of  embedded  feature  use.  Frequency  counts  were  automatically  collected  by  the  computer  from 
every  student  during  each  writing  session,  indicating  uses  of  note  cards,  mini-lessons,  branching  tools,  and  cut-and-paste 
procedures. 

Holistic  writing  assessment.  Holistic  ratings  of  story  quality  were  made  by  the  two  classroom  teachers  for  the 
two  stories  written  by  students.  The  following  holistic  scales  was  used: 

0.  Not  rated:  stories  were  absent,  unreadable,  or  unrelated  to  the  story  writing  task. 

1 . Unsatisfactory:  stories  were  abbreviated,  incomplete,  circular,  or  disjointed,  and  did  not  address  the  story- 
writing task. 

2.  Adequate:  some  elements  of  story  structure  were  presented  and  the  story  was  moderately  engaging. 

3.  Elaborated:  story  exceeded  the  essentials,  providing  additional  coherence  and  detail  to  create  an  interesting  and 

coherent  theme. 

To  establish  inter-rater  reliability  and  refine  the  rating  scale,  the  two  raters  initially  selected  at  random  three 
student  stories  from  the  entire  set,  scored  that  selection,  and  compared  their  ratings.  Variations  in  scores  were  followed 
by  discussion,  which  initially  prompted  the  decision  to  combine  two  categories  in  an  original  five-point  scale  into  one 
category  to  yield  the  above  four-point  scale.  Also,  additional  descriptors  were  added  to  categories  of  the  original  scale. 
Following  these  revisions,  eight  additional  stories  were  rated  independently  by  each  teacher,  with  a resultant  inter-rater 
reliability  of  r = .83.  The  teachers  again  discussed  any  discrepancies  to  increase  the  validity  of  subsequent  ratings. 
Overall  inter-rater  reliabilities  on  the  complete  story  sets  were  quite  high  for  both  Story  1 (n=  .94)  and  Story  2 (r  = .93). 
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Table  1. 

Summary  of  Data  Collection  Instruments  Used 


Instrument 

1 . Observer  Notes 

2.  Frequency  of  embedded 
feature  usage 


3.  Holistic  writing 
assessment 

4.  Student 

questionnaires/interviews 


5.  Teacher  interview 

6.  Problem-solving  analysis 

7.  Weekly  records 


Descriptor 

Each  day:  Written  descriptions  and  impressions  of  each  session 
were  recorded  in  a journal. 

Each  Day:  Computer-recorded  tabulations  of  the  number  of  times 
each  student  used: 

a)  Note  cards 

b)  Mini-lessons 

c)  Branching 

d)  Cut-and-paste 

Story  1 and  2:  4-pt.  rating  of  the  quality  of  students’  stories. 

a)  Weekl:  8 ratings  (5-pt.  scale)  on  attitudes  toward  and 
experiences  with  writing,  computers,  process  writing. 

b)  Week  6:  22  rating  items  on  attitudes  toward  and  usage  of 
embedded  features,  reactions  to  hypetext  and  its  effectiveness. 

c)  Week  6:  1 1 -question  interview  on  attitudes  and  experiences. 
Week  6:  17-questions  on  experiences  with  and  attitudes  toward 
the  hypertext  and  the  writing  process. 

Each  day: 

2 or  3 students  story  stacks  were  selected  for  qualitative  analysis 
Each  week:  Analysis  and  evaluation  of  students’  modeling 
strategies,  embedded  feature  use,  and  teacher-embedded 
instructions 


Student  questionnaires  and  interview.  Students  completed  two  questionnaires,  both  using  five-point  Likert-type 
scales  (1  ss  “strongly  disagree;1'  5 = “strongly  agree”)  for  responses.  The  initial  questionnaire,  administered  during  the 
first  week  of  the  study,  contained  eight  items  that  addressed  attitudes  about  writing  and  computers,  and  experiences  with 
the  process  approach  to  writing  (e.g.,  “I  often  share  drafts  of  what  I write  to  see  how  others  react  ).  The  second 
questionnaire,  administered  at  the  conclusion  of  the  study,  consisted  of  21  items  concerning  attitudes  about  writing, 
computers,  and  specific  hypertext  features  experienced  (e.g.,  “I  liked  using  the  branching  buttons  ). 

Ail  students  were  interviewed  during  the  final  week  of  writing  to  determine  how  they  used  and  felt  about  the 
hypertext  model.  There  were  1 1 interview  questions,  ranging  in  focus  from  general  impressions  (e.g..  Overall,  how  did 
you  like  the  hypertext  writing  environment?”)  to  reactions  about  specific  hypertext  features  (e.g..,  “What  usually 
prompted  you  to  use  the  mini-lessons?”). 

Teacher  interview.  The  two  teachers  were  interviewed  at  the  end  of  the  year.  Questions  addressing  the  following 
categories  were  asked:  reactions  toward  the  writing  environment,  student  attitudes,  ease  of  managing  the  writing 
environment,  the  instructional  effectiveness  of  the  writing  model,  most  and  least  liked  features,  and  suggested  changes. 

Daily  records  of  problem-solving  activity.  Two  or  three  student  story-writing  stacks  were  selected  randomly 
each  day  for  analysis  of  problem-solving  activity.  Based  on  qualitative  analyses  of  initial  samples,  the  following 
categories  were  derived  for  structuring  the  subsequent  evaluations:  generating  ideas,  expressing  ideas,  working  with  story 
structure,  branching,  originality,  developing  appropriate  details,  organizing,  and  “other.” 

Weekly  records.  Weekly  analyses  of  student  writings  were  conducted  using:  (a)  the  observer's  analysis  of 
modeling  strategies  used  by  the  student,  (b)  frequency  counts  for  the  use  of  each  embedded  feature,  (c)  descriptions  of  how 
students  used  embedded  features,  and  (d)  an  analysis  of  the  nature  of  teacher-embedded  instruction  or  help  on  story  writing 
strategies. 
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Procedure 


As  described  below,  the  implementation  of  the  hypertext  story  writing  program  followed  basic  methodologies  of 
the  formative  evaluation  model  suggested  by  Dick  and  Carey  (1991).  However,  since  the  main  purpose  of  the  study  was 
to  assess  the  classroom  implementation  as  it  might  be  used  by  typical  teachers  interested  in  a hypertext  process  writing 
application,  evaluation  procedures  evolved  according  to  teacher  interest  and  practical  considerations  as  well  as  procedures 
of  the  formal  Dick  and  Carey  model. 

Stage  One 

The  first  stage  assessed  ease  of  program  use,  employing  one  male  subject  (a  7th-grader)  who  was  not  part  of  the 
participating  seventh-grade  class.  Observer  notes  were  used  to  record  impressions  relevant  to  instructional  and  mechanical 
development  (e.g.,  ease  of  use,  perceived  effectiveness  of  different  components).  Journal  entries  included  the  recording  of 
subject  responses  to  the  writing  environment  via  “think  aloud”  procedures  (Schriver,  1989)  in  which  the  subject  orally 
described  his  thought  processes  as  he  worked  with  the  program.  After  completing  the  program,  the  subject  was 
interviewed  regarding  his  experiences  and  reactions.  Based  on  the  data  obtained,  appropriate  program  revisions  were  made. 

Stage  Two 

Stage  Two  used  a small  group  of  three  male  students  to  test  the  revised  program.  Procedures  were  similar  to 
those  described  for  Stage  One,  except  that  think  aloud  procedures  were  excluded.  Additional  revisions  of  the  program 
were  made  based  on  these  results. 

Stage  Three 

Stage  Three  was  the  actual  study,  conducted  with  the  targeted  junior-high  and  high-school  classes.  A six-week 
implementation  of  the  completed  hypertext  program  was  organized.  Week  1 was  initiated  with  the  administration  of  the 
questionnaire  to  assess  student  attitudes  toward  story  writing.  Students  received  instruction  from  the  first  author  on  using 
the  hypertext  story- writing  program.  Topics  included  opening  the  stack  and  leaving  the  stack,  turning  pages,  and  using 
embedded  features.  They  then  read  a hypertext  version  of  the  sample  story,  The  Frog  Princess.  They  were  also  asked  to 
read  the  mini-lessons  on  conferencing,  branching,  and  choosing  topics,  and  to  use  the  lesson  information  in  writing  a 
new  ending  to  The  Frog  Princess. 

The  students'  assignment  was  to  write  two  stories  on  any  topic  during  the  next  five  weeks.  Students  were  asked 
to:  (a)  record  goals  and  ideas  in  the  goal  buttons;  (b)  use  the  teachers'  and  other  students'  reactions  for  feedback,  and  record 
those  reactions  in  the  editor's  buttons;  (c)  use  the  branching  feature  for  at  least  one  part  of  the  story  (WW  students  were 
explicitly  required  to  branch);  (d)  provide  feedback  to  other  student  authors;  and  (e)  read  the  mini-lessons  as  they  were 
introduced  during  the  four-week  writing  period. 

During  this  period,  the  first  author  recorded  notes  daily  using  the  observer  notes  form.  Copies  of  student  stacks 
were  made  at  the  conclusion  of  each  session.  Weekly  records  of  individual  student  and  whole-class  usage  of  hypertext 
features  were  also  compiled.  At  the  completion  of  six  weeks,  the  follow-up  student  attitude  survey  was  administered 
during  a class  period  and  the  teachers  and  students  were  interviewed. 

Results 

The  results  are  presented  in  four  major  sections  addressing:  (a)  writing  activities  and  products,  (b)  teacher  and 
student  attitudes,  (c)  hypertext  facilitation  of  the  process  model,  and  (d)  observer  impressions.  Where  inferential  analyses 
were  conducted,  the  .01  probability  level  was  used  to  reduce  the  family- wise  error  rate. 

Writing  Activities  and  Products 

Story  Characteristics  and  Use  of  Branching  Applications 

A total  of  72  stories  were  written  by  the  38  student  participants.  The  average  story  was  approximately  10 
hypertext  pages  in  length,  roughly  equivalent  to  5 standard  double- spaced  manuscript  pages.  The  breakdown  of  story 
themes  included:  adventure-fantasy  (33%),  detective/mystery  (24%),  folk  tale  (7%),  personal  (7%),  soap  opera  (7%), 
family  (7%),  futuristic  (7%),  and  “no  recognizable  theme”  (7%). 
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Branching  was  included  in  stones  by  69%  of  the  overall  sample,  58%  of  the  7th-graders,  71%  of  the  9th-graders, 
and  93%  of  the  WW  students.  The  majority  of  branches  (57%)  were  used  to  provide  multiple  story  endings,  such  as 
“Click  Branch  A for  the  Happy  Ending,  Branch  B for  the  Sad  Ending.”  The  next  most  common  type  (35%)  was  used  for 
elaboration  (e.g.,  “Click  Branch  A if  you  want  to  learn  more.”).  The  remainder  (7%)  comprised  briefer  forms  of 
embellishment  or  motivational  enhancement. 

Story  Quality 

On  the  four-point  holistic  rating  scale  (0-3),  mean  scores  for  all  students  combined  were  2.01  on  Story  1 and 
1 .89  on  Story  2.  Thus,  the  typical  student  wrote  a story  that  included  some  or  most  major  elements  and  was 
“moderately”  engaging.  Less  than  10%  received  the  highest  rating  (3),  indicating  a more  complete  story  structure,  and 
sufficient  coherence  and  detail  to  create  an  interesting  and  engaging  theme.  ANOVAs  performed  on  the  holistic  scores 
using  class  and  gender  as  independent  variables  showed  neither  factor  to  be  significant  (both  p’s  > .05). 

Embedded  Feature  Use 

Table  2 provides  a summary  of  the  mean  frequencies  of  embedded  feature  use  for  each  group.  As  shown, 
students  tended  to  use  more  of  the  features  in  Story  1 than  in  Story  2.  The  most  frequently  used  options  were  teacher 
comments  and  student  comments;  least  frequently  used  were  cut-and-paste  and  branches. 


Table  2 

Hypertext  Writing  Environment  Feature  Use  Totals 


1 Story  1 

Story  2 

FEATURE 

total 

F 

mean 

SD 

med. 

total 

F 

mean 

SD 

med. 

Teacher 

comments 

283 

7.44 

2.95 

.5 

100 

1.16 

4.13 

1 

Student 

comments 

231 

6.08 

4.70 

5.5 

51 

1.34 

2.55 

0 

Goal/Idea 

button 

116 

3.05 

2.61 

2 

42 

1.11 

1.74 

0.5 

Branches 

63 

1.66 

1.73 

1.5 

43 

1.08 

1.55 

0 

Mini-lesson 

use 

114  1 

3.0 

2.50 

2 

26 

.68 

1.30 

0 

Cut-and-paste 

91 

2.40 

4.56 

0 

11 

.13 

.13 

0 

Pages  written*3 

227 

5.97 

2.95 

5 

157 

4.13 

3.31 

5 

aMean  is  calculated  by  dividing  the  total  frequency  by  the  number  of  stories, 
b Note  that  each  hypertext  page  is  roughly  equivalent  to  1/2  of  a standard  double-spaced  page. 


Comparisons  between  student  classes,  using  one-way  ANOVAs,  showed  significant  effects  on  Story  1 for  mini- 
lessons, E(2,  35)  = 8.14,  p<  .01;  cut-and-paste,  E(2, 35)  = 4.46,  p<  .01;  and  branching,  E(2,  35)  = 5.35,  p < .01. 
Follow-up  analyses  indicated  that  the  WW  students  (M  = 4.75)  more  frequently  used  mini-lessons  during  Story  1 than  did 
the  7th-graders  (M  = -50).  The  WW  students  also  used  cut-and-paste  tools  (M  = 5.33)  and  branching  (M  = 4.08)  more 
frequently  than  did  9th-graders  (M’s  = .50  and  1 .50,  respectively).  No  gender  differences  were  found.  Nor  were  there  any 
class  or  gender  effects  on  Story  2. 

Pearson  correlations  between  embedded  feature  scores  and  holistic  writing  scores  were  computed  to  explore 
whether  writing  quality  was  related  to  usage  of  any  of  the  6 features  on  either  story.  The  resultant  correlations  (n  = 12) 
were  all  positive  in  direction  but  nonsignificant  and  weak  to  moderately  weak  in  strength,  ranging  in  magnitude  from  .05 
(cut-and-paste  with  Story  1 score)  to  .38  (student  comments  with  Story  2 score). 


444 


Student  and  Teacher  Attitudes 


Initial  Student  Survey 

Student  responses  on  the  initial  survey  showed  moderately  positive  reactions  to  writing,  with  75%  of  the 
sample  agreeing  that  they  liked  writing,  60%  agreeing  that  they  liked  using  the  computer  for  writing,  only  20%  agreeing 
that  writing  was  difficult  for  them,  and  10%  agreeing  that  writing  is  difficult  for  others.  About  half  of  the  students 
reported  sharing  drafts  of  writing  with  others  and  thinking  about  reader  reactions  while  writing.  Relatively  few  (from  10- 
25%)  responded  that  they  used  the  work  of  others  while  writing  or  experimented  with  different  ways  of  writing  a story. 
None  of  the  chi-square  analyses  for  class  or  gender  was  significant  (all  p’s  > .01). 

Final  Student  Survey 

Table  3 summarizes  the  percentages  of  students,  broken  down  by  class,  agreeing  with  the  individual  item 
statements  on  the  final  survey,  and  the  item  chi-square  probabilities  for  class  and  gender  analyses.  The  only  significant 
chi-square  (atp  < .01)  result  was  for  class  (p  < .001)  on  Item  1,  “I  like  writing  stories.”  Ninth-graders  (19%)  were  less 
likely  to  agree  with  this  statement  than  were  7th-graders  (75%)  and  WW  (75%)  students.  In  general,  students  of  both 
sexes  in  all  grade  levels  had  a high  degree  of  confidence  in  their  own  and  other’s  writing  abilities,  as  indicated  by  the 
strong  majorities  (about  80%)  who  disagreed  with  the  statements,  “Writing  is  difficult  for  me”  (Item  3)  and  “Writing  is 
difficult  for  others”  (Item  4).  Approximately  two-thirds  (although  only  30%  of  9th-graders)  agreed  that  they  liked  using 
the  computer  for  writing. 

Medium  preferences.  Across  all  classes,  less  than  40%  of  the  students  agreed  that  they  would  rather  use 
HyperCard  than  word-processing  (Item  5).  On  the  other  hand,  only  about  25%  agreed  that  they  would  rather  use  a pen- 
and-pencil  (than  a computer)  to  write  stories  (Item  6)  and  about  60%  agreed  that  they  liked  writing  with  hypertext  (Item 
7). 

Ease  of  using  embedded  features.  Although  there  were  no  statistically  significant  patterns,  WW  students  tended 
to  be  slightly  more  positive  than  their  younger  counterparts  regarding  the  use  of  most  HyperCard  features  (branching, 
editor  buttons,  idea/goal  buttons,  and  cut-and-paste).  Five  survey  items  asked  students  to  rate  the  ease  of  using  the 
individual  embedded  features  (Items  8-12).  Overall  responses  indicated  that  from  about  60%  (for  mini-lessons)  to  80% 

(for  branching)  of  the  students  agreed  that  the  features  were  easy  to  use. 

Reactions  to  embedded  features.  Branching  was  the  most  liked  embedded  feature  (about  55%  overall  agreement; 
Item  13).  Editor  buttons,  goal  buttons,  and  especially,  mini-lessons  were  disliked  by  the  majority  of  students  (see  Table 
3,  Items  14-16).  The  cut-and-paste  feature  was  viewed  positively  by  75%  of  the  WW  students  but  by  only  50%  of  the 
9th-graders  and  31%  of  the  7th-graders  (Item  17), 

Frequency  of  using  embedded  features.  In  responding  to  Items  18-22,  students  indicated  generally  infrequent  use 
of  the  comment  buttons  (15%  agreement),  goal  buttons  (17%),  and  mini-lessons  (14%);  Slightly  more  frequent  use  of 
branching  buttons  (43%)  and  cut-and-paste  (55%)  was  reported. 

Attitude  changes  over  time.  Paired  ttests  were  performed  comparing  attitudes  toward  writing  before  and  after  the 
study.  The  first  four  ratings  items  (see  Table  3)  were  used  in  these  analyses.  None  of  the  comparisons  of  means 
approximated  significance  (lowest  p = .20) 
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Table  3. 

Percentages  of  Agreement  bv  Class  and  Chi-Square  Results  on  Einal  Student  Survey 


Class 

Chi-Square 

Probabilities 

Item 

7th 

9th 

Writer’s 

Workshop 

Class 

Gender 

1.  Like  writing  stories 

75 

10 

75 

.001 

.05 

2.  Like  using  computer  for  writing 

67 

30 

70 

.36 

.94 

3 . Writing  is  difficult  for  me 

19 

20 

25 

.92 

.51 

4.  Writing  is  difficult  for  others 

63 

20 

42 

.07 

.10 

5 . Rather  use  HyperCard  than  word  processing 

44 

40 

33 

.26 

.41 

6.  Rather  use  pen  or  pencil 

25 

30 

22 

.94 

.19 

7.  Like  writing  with  HyperCard 

63 

60 

50 

.79 

.92 

8.  Easy  to  use  branching 

88 

80 

83 

.87 

.20 

9.  Easy  to  use  editor  buttons 

44 

60 

75 

.25 

.59 

10.  Easy  to  use  goal  buttons 

81 

60 

75 

.48 

.32 

1 1 . Easy  to  use  mini-lesson 

56 

50 

70 

.44 

.68 

12.  Easy  to  use  cut-and-paste 

69 

70 

58 

.80 

.14 

13.  Like  branching 

56 

60 

67 

.85 

.69 

1 4 . Liked  editor  buttons 

25 

20 

58 

.10 

.19 

1 5 . Liked  idea/goal  buttons 

25 

20 

58 

! .98 

.59 

16.  Liked  mini-lessons 

25 

0 

17 

.23 

.38 

1 7 . Liked  cut-and-paste 

31 

50 

75 

.07 

.14 

18.  Frequently  used  branching 

31 

50 

67 

.17 

.28 

t9.  Freauentlv  used  comment  buttons 

25 

10 

33 

.43 

.35 

20.  Frequently  used  goal  buttons 

38 

0 

25 

.09 

.15 

2 1 . Frequently  used  mini-lessons 

18 

10 

33 

.39 

.49 

22 . Frequently  used  cut-and-paste 

44 

50 

75 

.25 

.02 

Student  Interview 

Interview  responses  corroborated  the  survey  results  in  all  cases.  In  discussing  their  prior  experiences,  few 
students  indicated  any  familiarity  with  process  writing,  while  about  50%  (more  so  for  WW  students)  indicated  having 
prior  computer  writing  experience.  About  half  indicated  that  they  liked  writing  stories. 

Positive  aspects  of  hypertext.  The  most  frequently  given  reasons  for  liking  HyperCard  were  branching  (L=  1 2), 
followed  by  ease  of  feature  use  (1=  10),  and  the  visual  aspects  of  the  environment  (f>  8).  Comments  about  branching 
generally  described  the  experience  as  creative,  a good  way  to  expiore,  and  unique:  6%  of  7th-graders,  50%  of  9th-graders, 
and  42%  of  WW  students  made  positive  comments  about  branching. 

Ease  of  use  was  described  mainly  in  terms  of  liking  the  tools  that  were  immediately  accessible  on  the  page,  not 
hidden  in  pull-down  menus.  Positive  comments  were  made  by  31%  of  7th-graders,  18%  of  9th-graders  and  0%  of  WW 
students. 

Visual  aspects  were  addressed  in  terms  of  how  the  environment  resembled  a book  or  page,  or  how  the  page  sizes 
were  small  and  unintimidating.  Positive  comments  were  made  by  19%  of  7th-graders,  30%  of  9th-graders,  and  0%  of 
WW  students. 

Negative  aspects.  The  most  frequently  given  reason  (f  = 4)  for  disliking  the  hypertext  writing  environment  was 
problems  associated  with  the  small  page  size.  As  one  WW  student  described  it,  “The  page  size  was  so  limiting,  rather 
than  being  able  to  just  write,  you  had  to  continually  stop  and  prepare  for  the  next  page.”  Another  student  compared  the 
limitations  to  “being  on  a slave  ship,  with  too  many  paddles  needed  to  move  things  forward.”  Olher  negative  aspects,  all 
expressed  by  WW  students,  were  associated  with  computer  problems  (f  = 2),  dislike  of  writing  (f  = 2),  and  branching 
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difficulties  (f  = 2).  For  example,  one  student  indicated  that  branching  bothered  her  because  it  complicated  the  writing  of 
endings  to  the  stories. 

The  view  that  hypertext  was  not  helpful  for  problem  solving  was  expressed  by  75%  of  7th-graders,  100%  of 
9th-graders,  but  only  about  33%  of  the  WW  students.  Specific  comments  by  the  two  younger  groups  conveyed  the 
general  feeling  that  they  did  not  have  writing  problems  that  needed  attention.  In  contrast,  the  WW  students  were  more 
self-critical  and  saw  the  helpfulness  of  certain  features,  particularly  idea  buttons  and  branching,  for  developing  and 
incorporating  ideas. 

Facilitation  of  the  process  model.  When  students  were  asked  specifically  about  idea  development,  more  7th- 
graders  (81%)  and  WW  students  (55%)  than  9th  graders  (20%)  indicated  that  the  hypertext  environment  was  helpful. 
Several  students  specifically  identified  the  idea  button  and  branching  as  useful,  but  provided  no  explanation  as  to  how. 
Negative  comments  stressed  the  spontaneous  cr  internal  nature  of  idea  development,  which  reduced  the  need  for  external 
aids. 

When  asked  directly  if  hypertext  facilitated  planning,  agreement  was  expressed  by  one-third  of  the  WW  students, 
but  by  only  one-fourth  of  7th-graders  and  none  of  the  9th-graders.  Almost  all  students  disagreed  that  the  hypertext 
writing  environment  facilitated  drafting,  which  was  not  surprising,  given  that  students  were  not  specifically  instructed  to 
begin  by  writing  a quick  or  “sloppy”  copy. 

In  response  to  a question  concerning  how  well  the  hypertext  environment  facilitated  revision,  about  half  of  the 
subjects  described  it  as  helpful  while  the  other  half  described  it  as  not  helpful.  Positive  responses  about  the  helpfulness 
of  embedded  feedback  were  given  by  nearly  all  of  the  9th-graders,  two-thirds  of  the  WW  students,  but  only  two  7th- 
graders.  Specific  comments  suggested  that  the  feedback  helped  in  making  editing  changes  on  mechanics  and,  to  a lesser  • 
extent,  story  strategies,  clarity,  and  organization. 


T eacher  interviews  conveyed  opinions  about  the  hypertext  environment  that  were  similar  to  those  of  the 
students.  Specifically,  the  computer  teacher,  like  her  7th-  and  9th~graders,  was  positive  about  hypertext  writing  and  cited 
branching  as  one  of  the  main  reasons.  The  WW  teacher,  like  her  class,  conveyed  reserved  feelings  about  hypertext  and 
identified  the  small  page  size  and  technical  difficulties  as  the  main  problems. 

Both  teachers  indicated  problems  with  the  small  writing  space  and  with  the  fact  that  the  writing  process  had  to 
be  monitored  continually  to  move  the  cursor  manually  to  new  fields  (a  limitation  of  the  interface).  Thus,  teachers’ 
concerns  were  less  with  the  features  of  the  process  writing  model  than  with  the  ease  of  using  the  hypertext  writing 
environment.  Other  identified  problems  were  the  requirement  for  students  to  work  from  back-up  disks  and  the  difficulty 
of  generating  print  copies  of  stories  because  of  the  time  required  to  print  the  writing  story  stacks.  Operational  factors 
associated  with  the  technology  over-shadowed  the  teachers’  interest  in  the  instructional  implications  of  the  model  for 
process  writing. 


Hypertext  Facilitation  of  the  Process  Model 


Writing  Samples  Analysis 

Analysis  of  the  47  randomly  selected  writing  samples  identified  eight  general  categories  of  problems 
experienced  in  all  grades.  Frequencies  and  brief  characterizations  of  the  categories  are  summarized  in  Table  4 (for  a more 
detailed  description,  see  Lohr,  1993).  The  most  common  problems  concerned  the  development  of  story  structure  and  of 
appropriate  details.  Despite  the  problems  detected  by  the  analysis,  only  three  students  embedded  comments  (in  the  idea 
box  or  elsewhere)  indicating  personal  awareness  of  difficulties.  Further,  there  was  no  incidence  of  peer  feedback,  although 
students  had  been  instructed  to  provide  and  obtain  it. 
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Table  4 

Categories  of  Problems  Identified  in  Daily  Collections  of  Randomly  Selected  Student  Writing 


Problem  Category 

Description 

Frequency 

Story  Structure 

Story  had  no  conflict  or  resolution 

11 

Development  of  appropriate 
details 

Details  included  either  confused  the 
reader  or  did  not  facilitate  story 
development 

7 

Organization 

Story  is  not  focused.  Too  many 
characters  introduced  too  quickly 

4 

Publishing 

Inordinate  attention  to  graphic 
quality  over  story  content. 

4 

One  incident  of  student  changing 
font  to  "Script”  to  maintain 
privacy. 

Originality 

Story  is  cliche  or  too  similar  to 
another  story. 

3 

Expressing  ideas 

Superficial  description 

2 

Getting  ideas 

Blank  pages 

2 

Branching 

Story  is  not  enhanced  by  branches 

2 

Modeling 

We  predicted  that  students  night  model  or  mimic  expert  writing  strategies  observed  from  the  teacher’s  writing 
from  the  embedded  story.  The  Frog  Princess  or  from  observations  of  other  students.  Although  some  modeling  was 
noted,  it  tended  to  be  at  a superficial  level  and  did  not  appear  to  reflect  specific  writing  strategies  designed  to  improve 
story  quality.  Both  teachers  observed  that  modeling  was  mostly  represented  in  imitating  story  themes  that  others  had 
used. 

Embedded  Feedback  Comments 

To  determine  the  nature  of  the  feedback  provided  by  teachers  (f  = 123)  and  students  (f  = 34),  the 
feedback  comments  were  analyzed  for  basic  theme.  Five  categories  emerged:  (1)  72  comments  about  writing  mechanics, 
(2)  35  comments  of  encouragement  ("Keep  up  the  good  work”),  (3)  25  comments  on  basic  story  writing  strategies,  (4)  17 
comments  on  improving  clarity  or  organization,  and  8 comments  (all  from  students)  that  were  silly  or  entertaining.  The 
WW  teacher  was  responsible  for  75%  of  the  comments  on  mechanics,  but  only  8%  of  the  encouraging  comments;  the 
computer  teacher  gave  22%  and  35%  of  the  comments  in  these  respective  categories. 

Observer  Impressions 

A journal  was  kept  by  the  first  author  while  observing  the  five-week  classroom  implementation.  Descriptions 
of  the  main  highlights  of  the  observations  are  reported  in  two  sections:  Introductory  Lessons  and  the  four  weeks  of  story 
writing  (for  more  detailed  descriptions,  see  Lohr,  1993). 
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Introductory  Lessons 

Students  in  all  classes  completed  the  Introductory  Lesson  and  the  Frog  Princess  writing  activities  in  four  50- 
minute  sessions.  Most  of  the  problems  encountered  during  this  period  concerned  the  interface  One  problem  was  the 
need  for  the  teacher  to  access  each  student’s  computer  to  check  individual  work,  a time-consuming  task.  This  was 
remedied  by  requiring  students  to  save  and  submit  their  work  on  3.5  in.  back-up  disks. 

Overall,  observations  indicated  that  most  students  had  little  trouble  learning  to  use  the  system.  While  nearly  all 
were  successful  in  writing  endings  to  the  Frog  Princess,  as  a rule,  the  older  the  class,  the  less  was  written.  The  WW 
students,  for  example,  often  wrote  endings  consisting  of  only  a few  sentences,  while  7th-  and  9th-graders  typically  wrote 
several  pages.  Although  students  were  reminded  each  day  to  read  the  appropriate  mini-lessons,  most  seemed  to  ignore 
these  aids. 

Hypertext  Story  Writing 

At  the  start  of  the  four-week  hypertext  writing  phase,  qualitative  differences  across  grade  levels  were  immediately 
noticed.  Within  a few  days,  7th-graders  were  focused  on  writing  and  only  occasionally  needed  disciplinary  reprimands. 
This  self-directed  behavior  persisted  for  a majority  throughout  the  four  weeks.  Frequently,  7th-graders  stayed  after  class 
to  complete  their  work  and  many  made  comments  about  their  progress  when  they  saw  the  teacher  outside  of  class.  The 
7th-graders,  however,  were  not  very  active  in  or  successful  at  providing  feedback  and  assistance  to  peers. 

The  high  school  students  were  more  restless  and  more  difficult  to  engage  in  the  writing  program.  In  direct 
contrast  to  the  7th-graders,  these  older  students  continually  engaged  in  conferencing  with  others,  so  much  so  that  the 
teacher  eventually  banned  conferencing,  feeling  that  such  interactions  were  exploited  too  frequently  for  social  exchanges. 
The  9th-graders  were  the  only  group  to  request  working  on  stories  in  pairs.  The  teacher  allowed  this  activity  for  one  of 
the  two  stories.  Its  effectiveness  was  difficult  to  determine,  although  there  was  much  talking  between  the  students  as 
they  worked. 

Teacher  Behaviors 

The  computer  teacher  was  clearly  more  positive  about  the  hypertext  writing  environment  than  was  the  high 
school  teacher.  The  former  took  a more  open  approach  to  the  program,  allowing  rules  to  evolve  as  needed.  She  imposed 
few  requirements  on  students,  treating  branching,  for  example,  as  an  option  rather  than  a necessary  story  feature.  Rather 
than  requiring  students  to  write  in  a folk-tale  adventure  theme  consistent  with  the  Frog  Princess,  she  felt  that  students 
should  be  able  to  choose  their  own  theme  and  genre.  The  computer  teacher  was  very  communicative  about  her 
impressions  and  feelings.  She  made  frequent  comments  that  the  writing  environment  was  working  well. 

When  the  first  author  visited  classes,  the  high  school  teacher  rarely  volunteered  cc  mments  about  the  program 
and  generally  seemed  somewhat  negative  in  her  attitudes  toward  the  program  both  in  class  and  in  interviews.  She  set 
more  rules  than  did  the  computer  teacher,  and  displayed  a strong  commitment  to  monitoring  adherence  to  the  rules.  Her 
basic  rules,  which  promoted  a relatively  mechanized  approach  to  process  writing  consisted  of  requiring  10-page  stories, 
creating  branches,  and  featuring  a certain  number  of  characters  in  the  stories.  She  started  ever)'  class  by  listing  the  rules 
and  reminding  students  that  their  work  would  be  graded.  Compared  to  the  computer  teacher,  she  spent  more  time 
carefully  reading  students’  stories  and  making  comments  about  their  construction  and  quality. 

Problems 

Throughout  the  hypertext  writing  experience,  various  problems  occurred.  Major  types  consisted  of: 

Limitations  and  “bugs”:  lack  of  spelling  check,  writing  fields  were  too  short,  cut-and-paste  tools  could  move 
only  four  lines. 

Disk  and  computer  failure:  3 out  of  22  computers  needed  repair;  6 out  of  37  floppy  disks  went  bad  (several 
students  lost  their  only  copies  of  work). 

Interface:  did  not  permit  floppy  disk  backups  to  be  made  from  files  stored  on  the  hard  drive,  did  not  show  the 
date  or  time  of  last  use  on  stored  files,  the  system  locked  when  disk  became  full. 

Unplanned  teacher  activities:  Too  much  time  devoted  to  Story  1,  leaving  less  time  for  Story  2;  limited  use  of 
embedded  comments;  insistence  on  students’  producing  print  copies  of  all  work,  which  took  considerable  time  due  to 
graphics;  failure  to  follow  procedures  of  process  writing;  failure  to  require/encourage  reading  of  the  mini-lessons. 
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Discussion 


Findings  are  discussed  below  in  relation  to  the  three  major  research  questions. 


Embedded  Feature  Use 

Overall,  most  students  felt  that  the  embedded  features  (branching,  goal/idea  buttons,  editor  buttons,  mini- 
lessons, and  cut-and-paste)  were  easy  to  use.  Their  usage  of  those  features,  however,  tended  to  be  limited.  Three  reasons 
are  suggested.  The  first  was  lack  of  classroom  writing  time.  For  example,  when  students  were  asked  why  they  did  not 
use  the  mini-lessons,  a typical  response  was  “deadlines/too  busy.” 

A second  probable  reason  was  lack  of  familiarity  with  the  process  writing  concept  and  procedures.  Contrary  to 
teacher  claims,  almost  all  students  indicated  that  the  hypertext  experience  was  their  first  exposure  to  a process  orientation. 
Most  also  indicated  that  they  received  more  feedback  in  the  hypertext  writing  program  than  in  any  other  school  writing 
experience. 

A third  probable  reason  for  the  low  feature  use  was  lack  of  metacognitive  skills  to  assess  learning  needs.  When 
learning  tasks  are  difficult,  as  in  the  case  in  writing,  which  depends  heavily  on  metacognitive  skills  (Brown,  1987), 
learners  may  select  less  instructional  support  to  complete  the  task  earlier  or  because  they  don’t  see  its  value  (Ross  & 
Morrison,  1989). 

Problem-solving.  There  was  little  evidence  that  students  capitalized  on  the  hypertext  embedded  features  to  solve 
problems  while  writing.  Specifically,  few  students  used  branching  as  a way  to  develop  or  explore  different  arguments  in 
the  form  of  alternative  plots  or  story  endings.  Further,  with  the  exception  of  several  students  in  the  WW  class,  little 
attention  appeared  to  be  given  to  problem-solving  in  general,  a main  component  of  process  writing. 

Expert  modeling.  Overall,  modeling  was  limited  in  frequency  and  restricted  to  superficial  features,  such  as  story 
theme.  Based  on  our  observations  and  the  interview  responses,  students  apparently  found  it  too  time-consuming  and 
difficult  to  try  to  model  actual  story  writing  strategies. 

Non-linear  recursive  writing.  Findings  did  support,  at  a relatively  superficial  level,  facilitation  by  the  embedded 
features  of  planning,  revising,  and  editing  components  of  non-linear  writing.  With  regard  to  planning,  students 
embedded  comments  in  :Jea/goal  buttons  tended  to  be  summary  descriptions  of  story  content  rather  than  insights  into 
dilemmas  or  plans  of  the  writer.  The  implication  is  that  students  may  not  have  known  how  to  proceed  with 
communicating  their  ideas,  or  they  may  have  lacked  generation  and  development  skills.  According  to  Resnick  (1987), 
problem-solving  models  often  fail  because  students  don’t  know  how  to  apply  the  models  in  specific  contexts. 

The  high  school  students  made  greater  use  of  embedded  feedback  for  revision  than  did  7th-graders.  This  result  is 
not  surprising,  since  the  high  school  teacher  used  the  feedback  more  for  task-oriented  guidance  and  correction,  whereas  the 
computer  teachtr  used  it  more  for  reinforcement.  Based  on  student  reactions,  the  former  type  (academically-oriented) 
feedback  was  viewed  as  more  valuable. 

For  all  classes,  none  of  the  students’  embedded  comments  offered  help  to  others  in  story  writing  strategies. 

Some  story-writing  comments  were  made  by  the  computer  teacher  and  especially  by  the  high  school  teacher.  Again,  the 
latter’s  expertise  in  composition  appeared  to  be  the  critical  factor. 

As  the  above  results  convey,  although  the  embedded  features  were  logical  instructional  design  components,  they 
were  not  utilized  as  anticipated  to  support  the  process  writing  model.  The  mere  packaging  of  powerful  instructional 
strategies  in  technologically-sophisticated  media  presentations  does  not  guarantee  that  such  strategies  will  be  accepted  by 
teachers  or  learners.  For  example,  in  a study  of  student  uses  of  data-bases,  Neuman  (1993)  found  that  numerous 
inappropriate  design  features  of  the  data-bases  prohibited  easy  access  to  material  and  created  resistance  to  using  the  data- 
base systems.  Gallo  and  Horton  (in  press)  found  similar  problems  in  teachers’  usage  of  Internet  for  educational 
functions.  For  new  media  applications  to  be  effective,  the  status  and  needs  of  learners  (and  their  teachers)  must  be 
considered  as  strongly  as  the  content  design  features  (Jonassen,  Campbell,  & Davidson,  1994).  Two  important  factors 
are  therefore  likely  to  be  time  for  users  to  become  familiar  and  experienced  with  the  application  and  sufficient  front-end 
and  ongoing  training  to  facilitate  skill  acquisition. 

Writing  Improvement 

Moderate  to  low  writing  scores  in  the  holistic  evaluation  indicated  that,  in  spite  of  hypertext-based  efforts  to 
improve  the  more  nebulous  qualities  of  writing  (c.g.,  complexity,  organization,  clarity,  and  style;  see  llillerich,  1985), 
student  writing  in  this  study  tended  to  reflect  the  types  of  problems  suggested  in  prior  research  (Applebee  et  al.,  1990; 
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McGee  & Richels,  1990;  Stein,  1987).  Such  problems  included  lack  of  complex  plots  or  analyses  of  different  points  of 
view,  use  of  stereotypical  solutions,  and  limited  use  of  planning  and  writing  strategies. 

Analyses  of  writing  scores  indicated  that  the  quality  of  stories  did  not  improve  over  time,  as  might  be  expected 
from  an  environment  designed  to  facilitate  the  writing  process.  One  consideration  is  the  possible  insensitivity  of  the 
writing  assessment  for  detecting  changes  in  quality  over  an  eight-week  time  frame.  Limited  interventions,  like  the 
process  strategies  taught  here,  may  provide  useful  orientations  and  beginning  experiences  that,  with  additional  practice 
and  “incubation”  time,  engender  improved  skills.  Further  research  that  evaluates  the  strategies  over  an  extended  time 
period  is  therefore  suggested. 

Gender  and  Class  Differences 

Generally,  the  present  results  revealed  no  reliable  differences  between  male  and  female  students  in  their  usage  of 
the  system,  attitudes,  or  performance.  Class  differences  were  more  noticeable  across  the  various  outcome  measures. 
Although  the  majority  of  students  were  positive  about  the  writing  environment,  the  WW  students  were  consistently  less 
so  than  the  younger  students.  This  outcome  appears  mostly  attributable  to  two  factors,  the  students*  task  goals  and  the 
teacher’s  attitudes. 

For  the  WW  students,  the  task  goals  were  more  directed  toward  learning  to  write  than  to  increasing  their 
experiences  with  technology.  Technical  problems,  in  addition  to  the  small  page  size  which  limited  writing  space  and 
speed,  were  considered  hindrances  to  efficient  composition.  Further,  the  isolation  of  process  model  components  into 
separate  pop-up  buttons  and  spaces  may  have  inadvertently  worked  to  make  the  writing  process  more  linear  rather  than 
less  so.  By  separating  the  writing  space  for  goals  and  ideas,  idea  generation  may  have  been  forced  out  of  its  relevant  or 
natural  contexts.  One  12th-grader,  for  example,  indicated  that  preparing  to  record  thoughts,  rather  than  just  writing  the 
thoughts  into  his  work,  served  to  interrupt  writing  rather  than  facilitate  it. 

The  more  positive  attitudes  by  the  younger  students  seems  to  be  at  least  partly  attributable  to  visual  features. 
Several  students  elaborated  about  the  appeal  of  the  “book-like”  appearance  and  the  small  page  size,  which  some  of  the 
WW  students,  in  contrast,  described  as  “babyish.”  It  also  seems  important  that  the  younger  students  had  much  less 
experience  with  word  processing  and,  consequently,  had  fewer  expectancies  for  features  that  increased  convenience  and 
efficiency. 

The  contrasting  attitudes  of  the  two  teachers  suggests  the  importance  of  social  variables  in  the  acceptance  and 
implementation  of  technology,  a major  finding  in  word-processing  research  (Cochran-Smith,  1991).  The  high  school 
teacher  appeared  to  focus  more  on  the  appropriateness  of  the  hypertext  environment  for  production  rather  than  for  the 
development  of  writing  skills.  Accordingly,  she  viewed  the  system  as  less  “convenient”  than  conventional  word 
processing  for  creating  and  disseminating  student  work.  This  negative  attitude  toward  the  technical  qualities  of  hypertext 
was  mirrored  by  her  students,  an  outcome  similar  to  the  one  found  by  Bradley  (1993)  in  her  study  of  factors  influencing 
student  attitudes  toward  Channel  One. 

The  computer  teacher,  in  direct  contrast,  took  great  interest  in  the  potential  of  hypertext  to  stimulate  new, 
creative  approaches  to  writing.  She  perceived  the  various  embedded  features  as  tools  for  exploring  different  approaches  to 
writing  rather  than  as  impediments  to  efficient  production  of  final  drafts.  As  a result,  her  students  appeared  much  more 
open  to  using  the  system,  although,  like  the  high  school  students,  they  quickly  abandoned  features  that  they  found  less 
relevant  to  their  personal  interests  and  objectives.  The  clear  implication,  however,  is  that  for  technology  to  have  an 
impact  on  classroom  instruction,  teachers  must  view  the  new  systems  as  beneficial,  be  invested  in  their  application,  and 
be  sufficiently  trained  in  their  use  (Gallo  & Horton,  in  press;  also  see  Garland,  1991).  In  the  present  study,  what  was 
thought  to  be  reasonable  time  working  with  teachers  was  apparently  insufficient  for  achieving  these  skills  and 
dispositions. 

Recommendations 

While  technological  problems  may  reduce  the  success  of  new  computer-based  instructional  strategies,  factors 
intrinsic  to  the  subject  area  or  strategies  themselves  may  prove  just  as  limiting.  In  this  regard,  barriers  to  the  success  of 
the  present  hypertext  environment  proved  to  be  problems  intrinsic  to  process  writing  instruction,  as  identified  in  the 
literature  (Applebee  et  al.,  1990;  Calkins,  1986;  Stein,  1987).  Students  were  unfamiliar  with  the  process  orientation,  the 
instruction  provided  by  teachers  was  superficial,  and  students*  story  writing  skills  were  generally  poor.  While  the  various 
support  features  (idea  buttons,  editor  buttons,  branching,  etc.)  were  consistent  with  process  writing  strategies,  technical 
problems  and  teacher  inexperience  limited  their  utility.  Based  on  the  results,  some  suggestions  for  future  instructional 
design  in  this  area  are  offered  below. 
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As  stated  by  Jones,  Li,  and  Merrill  (1992),  HyperCard  is  a type  of  rapid  prototype  model  designed  more  for 
programmers  than  for  designers.  Other,  more  customized  prototypes  may  represent  more  convenient  models  because  they 
consist  of  components  related  to  instructional  design  needs,  and  are  therefore  not  only  easier  to  assemble  quickly,  but 
address  the  needs  of  the  content  more  efficiently.  Thus,  while  the  hypertext  environment  presents  new  possibilities  for 
creative,  non-linear  story  writing,  its  lack  of  adaptive  features  for  supporting  such  writing  applications  leads  to  the 
typical  problems  that  occur  when  instruction  is  driven  by  the  available  technology  rather  than  by  the  needs  of  the  learner 
(Davies,  1993;  Newman,  1993).  Since  the  HyperCard  environment  had  limited  text  editing  procedures,  it  was  not 
feasible  to  revise  the  stack  and  conduct  further  testing  to  refine  the  product  as  recommended  by  Dick  & Carey  (1991 ). 
Ideally,  we  would  have  proceeded  to  the  development  of  an  application  to  address  these  shortcomings  with  a more 
powerful  development  environment.  This  development  effort,  however,  was  beyond  the  scope  of  this  project. 

In  the  present  study,  two  major  categories  of  problems  related  to  operational  difficulties  and  weaknesses  in  the 
instructional  design.  It  would  should  also  be  noted  that,  even  though  Stage  1 and  Stage  2 of  the  evaluation  model  were 
not  as  rigorous  as  their  counterparts  in  the  Dick  and  Carey  (1991)  formative  model,  these  phases  were  unlikely  to  identify 
many  of  the  problems  that  actually  emerged  when  the  hypertext  instructional  program  was  actually  integrated  with 
teacher  instruction,  class  goals,  and  other  school  or  class  activities.  Possible  improvements  for  process  writing 
applications  are  suggested  below. 

1.  Make  the  system  operate  more  like  a word  processor,  to  include  spell  checking,  more  space  for  cut-and-paste, 
and  repositioning  of  the  cursor  at  the  end  of  a page. 

2.  Improve  the  security  interface  (At  Ease)  to  allow  for  easier  file  management  and  protection  of  individual 

files. 

3.  Allow  for  networking  arrangements  to  facilitate  communication  exchanges  and  file  access  between  students 
and  the  teacher. 

4.  Give  teachers  more  control  over  the  writing  environment  by  either  requiring  students  to  read  the  mini-lessons 
or  teaching  the  declarative  and  procedural  knowledge  via  other  instructional  methods.  As  students  gain  more  experience 
and  demonstrate  mastery  of  process  writing  skills,  they  should  be  given  more  freedom  to  experiment  freely,  even  by 
creating  multiple  levels  of  branches.  This  recommendation  of  graduated  learner-control  allowances  is  consistent  with 
conclusions  from  other  learner-control  studies  (Ross  & Morrison,  1989;  Tennyson,  1981). 

5.  Program  the  hypertext  environment  to  require  students  to  complete  certain  units  or  master  certain  skills 
before  they  are  allowed  access  to  certain  features.  Additional  programming  might  also  be  used  to  provide  teachers,  for 
monitoring  purposes,  descriptive  summaries  of  student  feature  uses  and  writing  activities. 

6.  Explore  means  of  using  branching  more  effectively.  Branching  was  the  most  popular  program  component 
and  the  one  most  frequently  cited  by  students  in  discussing  idea  generation  and  development.  Branching  also  appears  a 
highly  appropriate  format  for  introducing  complexity  and  using  critical  thinking  skills  in  writing  (see  Resnick,  1987).  A 
possible  refinement  of  the  present  instructional  strategy  would  be  to  include  more  direct  teaching  and  modeling  (through 
story  samples)  of  effective  branching  strategies. 

7.  Give  more  attention  to  developing  teacher  expertise  and  interest  in  using  the  hypertext  system. 

Of  all  the  variables  investigated  in  this  study,  most  critical  to  the  success  of  the  hypertext  writing  environment 
appeared  to  be  those  relating  to  teacher  and  student  perceptions  of  its  efficiency.  The  results  also  support  general  findings 
of  other  research,  which  identify  the  highly  influential  role  of  classroom  social  systems  on  the  success  of  any  new 
technology  (Cochran-Smith,  1991;  Reilly,  1992).  The  implication  is  that  it  makes  little  sense  to  overlay  such 
technologies  on  existing  instructional  systems  and  expect  immediate  acceptance  and  operational  efficiency.  The 
continuing  challenge  for  instructional  designers  is  effectively  integrating  the  new  delivery  strategies  to  lit  classroom 
conditions  and  curriculum  needs.  As  demonstrated  in  this  research,  systematic  evaluation  can  provide  a valuable  tool  for 
facilitating  this  integration  over  time. 
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Abstract 


High  tech  organizations  such  as  Storage  Technology  Corporation  (STK)  are  increasingly  under  pressure  to  both 
reduce  cycle  time  in  bringing  newer  technologies  to  an  international  market  place  and  produce  timely  and  effective 
instruction.  Traditional  cascaded  Instructinal  Systems  Design  Models  (ISD),  assumed  to  be  in  common  practise  are  used 
as  processes  at  STK.  A survey  was  distributed  to  determine  which  steps  are  followed  in  an  ISD  model  and  why  certain 
steps  are  ommitted.  Inquiry  findings  indicate  that  the  models  are  not  followed  because  employees  do  not  understand  all 
the  steps,  do  not  have  the  time  to  complete  them,  or  decisions  have  been  preempted  prior  to  the  project  getting  to  the 
instructional  designer.  The  ability  of  ISD  models  to  meet  instructional  designer  needs  is  discussed  in  light  of  the  inquiry. 

Introduction 

Current  Instructional  Design  Models,  defined  here  as:  traditional  cascaded  models  (Fig.  1),  relying  on  formal 
rules  and  linear  steps,  are  assumed  to  be  commonly  applied  in  corporate  educational  departments  (Hannum  & Hansen, 
1989).  Proponents  of  these  models,  such  as  Hannum  & Hansen,  Dick  and  Carey,  and  others,  have  adapted  a systems 
approach  to  instructional  design.  The  outcome  of  this  adaptation  posits  an  increasing  use  of  formal  instructional  systems 
design  (ISD)  models  in  corporations  (Hannum  & Hansen,  1989). 
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Figure  7 The 


Dick  & Carey  ISD  Model 


The  role  of  technology  in  the  corporate  world 


Technology  in  the  corporate  world  continues  at  an  unprecendented  expansion  since  ISD  models  came  into 
widespread  use.  Along  with  technological  advancement,  there  exists  a coevolving  use  of  technological  tools  designed  to 
facilitate  use  of  the  new  technologies.  Today’s  corporation  presents  us  with  unprecedented  complexity.  Such  complexity 
produces  a need  to  provide  timely  training,  not  only  of  the  technologies  themselves,  but  also  of  the  tools  which  provide 
access  to  using  the  technologies. 

The  effects  of  technological  change  spill  over  to  corporations  which  may  not  necessarily  think  of  themselves  as 
“technology-based.’’  Examples  extend  to  workers  needing  to  learn  new  phone  systems,  email,  word  processing,  data 
bases,  and  a myriad  of  other  technological  tools  which  permeate  the  modern  corporation. 

Cross-domain  knowledge  thus  becomes  an  essential  component  for  workers.  They  must  not  only  cope  with  the 
need  to  understand  their  primary  job  function,  but  also  must  acquire  whatever  corollary  knowledge  of  tools  deemed 
necessary  to  “do  their  job.’’ 

While  rapid  enhancements  in  a wide  range  of  technologies  drive  rapid  turnover  of  new  products,  they  also  create 
an  easily  reached  global  marketplace.  The  global  marketplace  increases  competition  and  subsequently  generates  a need  to 
reduce  cycle  time  for  new  product  development.  Reduction  of  cycle  time  extends  to  include  instruction  and  training.  The 
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challenge  of  producing  effective  instruction,  on  a timely  basis  by  corporate  education  departments,  increases 

pr<  portionately  with  the  technological  change.  An  ever  more  rapid  cycle  of  new  product  development  drags  with  it  a need 

for  timely  new  product  training. 

The  influence  of  standards  criteria 

By  the  year  1995,  the  European  Union  (EU)  imposed  the  requirement  of  meeting  ISO  9000  Standards  as  a 
prerequisite  for  conducting  business  within  its  member  countries.  ISO  9000  st  ndards  brought  Total  Quality  Management 
(TQM)  to  all  corporations  conducting  business  with  the  EU.  Therefore,  ISO  9000  standards  provided  a measurable 
standard  of  quality  for  any  corporation  doing  business  with  the  EU.  Corporations  needed  “certification”  by  an 
independent  agency,  which  stated  the  corporation  met  the  standards.  These  independent  agencies  conduct  yearly  audits  of 
certified  corporations,  ensuring  continued  compliance  with  the  ISO  9000  standards. 

A great  deal  of  time  and  effort  has  been  put  forth  by  multinational  corporations  to  comply  with  the  standards.  A 
component  of  the  standards  is,  “...quality  education  and  training”  (Scherkenbach,  1990).  In  effect,  corporations  must  not 
only  have  education  plans  in  place  for  their  employees,  but  must  also  comply  with  a rigid  process  for  the  design, 
development,  and  delivery  of  such  education.  On  the  surface,  ISD  models  seem  to  represent  the  perfect  marriage  of 
meeting  ISO  standards  and  producing  effective  instruction. 

Problem  definition 

Given  the  need  to  “reduce  cycle  time”  while  at  the  same  time  cleaving  to  ISO  9000  standards,  how  do  corporate 
education  departments  fare  in  producing  quality  as  well  as  timely  instruction? 

Research  Question 

Before  a measurement  of  how  well  do  corporate  education  departments  meet  the  needs  of  their  customers  can  be 
made,  a research  question  centering  around  how  the  departments  go  about  producing  instruction  must  be  answered.  The 
primary  motivation  for  this  study  at  Storage  Technology  Corporation  becomes,  “How  does  the  process  of  instructional 
systems  design  proceed  at  Storage  Technology  Corporation  (STK)?” 

Prior  studies 

The  theoretical  grounding  which  underlies  this  inquiry  concerns  accepted  instructional  design  models  such  as 
those  of  Dick  and  Carey  (1990).  These  models  have  long-standing  application  in  both  industry  and  in  the  military. 
Recent  publications  (Wedman  & Tessmer,  1990  and  Edmonds,  Branch  & Mukeijee,  1994)  have  sought  to  address  the 
inadequacy  of  traditional  design  models. 

A number  of  authors  have  recognized  the  limitations  of  ISD  models  in  fast-paced  environments,  or 
environments  where  cultural  factors  interject  themselves  in  the  design  process.  Wedman  and  Tessmer  (1990)  describe  the 
potential  for  erosion  of  educational  quality  by  haphazardly  omitting  steps  from  a design  model.  They  propose  a “layers 
of  necessity”  model  which  offers  a set  of  heuristics  by  which  steps  can  be  omitted  without  completely  destroying  the 
quality  of  the  instruction. 

Edmunds,  Branch  and  Mukeijee  (1994)  offer  a framework  for  comparison  of  different  mstructional  design 
models,  with  the  idea  of  choosing  a model  which  “best  fits”  the  situation.  Their  comparison  framework  of  the  different 
models  currently  in  use  permits  a reasonable  assessment  of  which  models  to  use  for  differing  design  needs,  and  a measure 
of  the  model's  chance  for  being  reliably  successful,  given  different  types  of  instruction  (classroom  focused,  product, 
focused,  system  focused). 

Rowland  (1992)  attempted  a definition  how  “expert”  instructional  designers  and  “novice”  instructional  designers 
approached  the  solution  of  an  instructional  problem.  Large  differences  in  approaches  to  instructional  design  problem 
solving  emerged.  Significantly,  experts  formed  “mental  models”  and  looked  for  causal  relationships  and  deep  system 
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understanding.  They  formulated  solutions  to  different  links  within  the  problem,  and  adopted  a variety  of  possibilities  for 
those  solutions.  Their  problem  interpretation  was  categorized  as  ill-defined,  in  that  they  were  rather  flexible  in  adopting 
and  discarding  multiple  solutions.  In  all  cases  they  used  lengthy  analysis,  multiplicity,  and  variety. 

The  above  approach  sharply  contrasted  to  novices,  who  “believed”  the  information  they  were  given  and 
performed  shallow  analyses  and  referring  to  the  problem  information  that  was  given  them  at  frequent  intervals. 

The  basic  premise  put  forth  by  Wedman  and  Tessmer  (1994)  is  that,  “Design  models  are  based  on  assumptions 
which  are  incompatible  with  practice.”  Wedman  and  Tessmer  suggest  the  incompatibility  exists  because  the  design 
models  do  not  permit  “selective”  exclusion  of  particular  steps. 

Since  in  their  study  steps  were  clearly  omitted,  then  quality  of  instruction  could  be  eroded  due  to  the  lack  of  any 
prescribed  and  analytical  way  to  exclude  such  steps.  Their  study  surveyed  73  training  professionals  involved  in 
instructional  design.  The  survey  was  a two  part  survey.  The  first  part  of  the  survey  asked  the  frequency  with  which  the 
subjects  completed  eleven  steps  in  the  ID  model  which  are  traditionally  thought  of  as  critical  to  the  process.  The  second 
part  requested  die  subjects  to  give  reasons  why  a particular  design  activity  was  excluded. 

The  implication  of  Wedman’s  & Tessmer’s,  and  Rowland’s  findings  for  the  case  of  multinational  corporations 
lies  within  the  notion  that  the  bulk  of  their  technical  trainers  come  from  a technical  background,  with  little  or  no 
background  in  instructional  design 

The  question  must  be  raised,  “Does  instructional  design  from  such  persons  suffer  from  their  lack  of 
understanding  of  the  process  of  ISD?”  From  this  question  the  assumption  that  instructional  design  models  are  indeed 
followed  during  the  production  of  instruction,  needs  :*  comprehensive  evaluation,  given  the  current  corporate 
environment,  with  its  cycles  of  technological  change,  requirements  to  reduce  cycle  time,  and  pressures  of  global 
competition. 


Method 


Storage  Technology. Corporation  target  population 

Storage  Technology  Corporation  (STK)  is  a producer  of  hardware  and  software  within  the  mainframe,  mid-range 
(small  mainframes),  and  open  systems  (networked)  markets.  They  are  currently  positioning  themselves  to  enter  the  open 
systems  market  to  a greater  extent.  STK  is  a multinational  corporation,  which  does  some  40%  of  its  business  in  foreign 
markets.  As  such  it  fits  the  framework  outlined  above  and  is  ISO  9000  certified.  The  major  products  STK  has 
historically  excelled  in  have  been  storage  related,  comprised  mainly  of  high  speed  tape,  and  solid  state  storage  systems  in 
the  terrabytc  range  and  attached  to  large  main  frame  computing  environments. 

Spheres  of  responsibility  within  STK  Headquarters  Corporate  Education  comprise  five  subsets,  plus  adjunct 
positions  directly  under  the  Director  of  Corporate  Education.  The  five  subsets  follow: 

* Training  Technology  (Multimedia) 

* Hardware  Education 

* Software  and  Marketing  Education 

* Headquarters  Education 

* Leadership  and  Quality  Education 

Each  department  within  Corporate  Education  encompasses  different  arenas  of  instruction.  In  addition  there  are 
satellite  regional  education  offices  within  the  US  and  other  countries.  As  a result  one  of  Corporate  Education’s  funciions 
is  that  of  “train  the  trainer,”  i.c.  to  teach  those  who  return  to  their  respective  locations  to  teach. 
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Subjects 


A total  of  67  subjects  were  included  in  the  inquiry.  Baseline  information  about  these  subjects  is  limited  and 
currently  generalizable  to  stating  that  they  have  diverse  education  and  backgrounds,  regarding  both  ISD  and  their  subject 
content  areas.  Job  responsibilities  seem  to  reflect  this  diversity,  ranging  from  being  involved  in  the  entire  process  of 
ISD,  plus  delivery  (teaching),  to  being  involved  in  just  a few  of  the  component  steps,  i.e.  just  delivery  or  just  the  design 
phase,  or  development  phase.  For  the  purpose  of  this  study,  delivery  will  be  included  in  the  ISD  process. 


The  inquiry  utilized  a survey  adapted  from  Wedman  & Tessmer  (1994)  which  was  based  on  Dick’s  and  Carey’s 
(1990)  ISD  model.  The  survey  probed  which  ISD  activities  get  completed,  which  activities  are  omitted,  and  why  the 
activities  are  omitted.  The  activities  covered  the  same  territory  as  Wedman  and  Tessmer  (1993)  and  are  outlined  below: 

• conduct  a needs  assessment 

• training  is  the  solution 

• conduct  a task  analysis 

• write  learning  objectives 

• identify  learning  outcomes 

• assess  student  entry  level  skills 

• develop  test  items 

• select  instructional  strategies 

• select  media  formats 

• pilot  test  instructions 

• do  formative  evaluation 

Frequency  of  inclusion  in  the  ISD  process  again  followed  Wedman  and  Tessmer  and  was  expressed  as,  Not 
applicable  (NA),  Always , Usually , Occasionally , Never.  In  addition  we  asked  if  there  were  design  activities  which  the 
respondents  did  which  were  not  addressed  in  the  survey. 

A second  part  of  the  survey  listed  reasons  which  were  germane  at  STK  about  why  certain  activities  were  not 
completed.  They  are  listed  below: 

• not  perc  ;ved  as  applicable  (NA) 

• lack  expertise 

• decision  already  made 

• unnecessary 

• no  time 

• no  money 

• customer  will  not  permit  inclusion 

• restriction  on  travel 

• no  process  in  place 

A third  portion  probed  respondents  background,  including  their  education  in  ISD,  their  education  for  the  content 
area  within  which  they  produce  instruction,  their  experience  with  ISD,  any  prior  experience  or  formal  instruction  in 
teaching.  In  addition  there  were  open-ended  portions  which  probed  perceived  problems  with  STK’s  apprach  to  ISD  and 
solicited  input  as  to  how  to  change  direction. 

Baseline  information  therefore,  addressed  what  combinations  of  instructional  design  experience  and  education  in 
instructional  design  the  respondents  bring  to  the  department.  It  also  addresses  what  subject  matter  expertise  respondents 
bring  to  their  design  activities. 
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Analysis 


There  were  a total  of  46  persons  who  responded  to  the  survey  (69%).  Of  the  46,  8 (17%)  had  formal  university 
level  courses  in  instructional  design.  Only  4 (9%)  had  a degree  or  a major  area  of  study  in  instructional  design.  18  or 
39%  had  graduate  degrees,  16  or  35%  had  undergraduate  degrees,  2 or  4%  had  associate  degrees  and  6 or  13%  had  high 
scool  diplpomas.  The  group  with  the  lowest  number  of  degreed  personnel  were  in  the  hardware  education  group  (3 1 %), 
the  highest  number  were  in  the  software  engineering  group  (100%). 


Table  1 : numbers  and  percentages  of  respondents  with  IP  course  work 


Ed  Departments  @stk 

Number  & percentage  of 
respondents  with  IT  course  work 

Training  & Technology  Group 

4 57% 

Hardware  Education 

2 14% 

Software  & Marketing 

1 16% 

Leadership  & Quality  Education 

2 40% 

Headqarters  Education 

2 11% 

Management 

1 33% 

It  should  be  noted  that  the  above  percentages  of  individuals  with  IT  course  work  would  drop  if  those  who  did 
not  respond  to  the  questionnaire  had  done  so  (informal  conversation  with  members  of  the  department).  The  above  table 
also  refers  to  prsons  who  have  had  course  work  in  IT.  The  number  of  persons  holding  degrees  in  IT  at  STK  Corporate 
Education  is  actually  four. 

Results 


Table  2 Percentage  of  design  activities  completed  by  the  respondents 


Activity 

always 

usually 

occasionally 

never 

conduct  needs 
assessment 

27% 

39% 

24% 

9% 

training  is  the 
solution 

12% 

51% 

9% 

28%  1 

conduct  a task 
analysis 

42% 

27% 

27% 

4% 

write  learning 
objectives 

66% 

33% 

0% 

1% 

identify 

learning 

outcomes 

27% 

36% 

15% 

12% 

assess  entry 
level  skills 

19% 

22% 

36% 

22% 

develop  test 
items 

4% 

33% 

39% 

24% 

select 

instructional 

strategies 

27% 

33% 

33% 

7% 

select  media 
formats 

26% 

26% 

36% 

12% 

pilot  test 
instructions 

44% 

15% 

29% 

12% 

conduct  form- 
ative evals. 

25% 

30% 

30% 

15% 

0 
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The  above  results  loosely  follow  those  of  Wedman  and  Tessmer  (1993),  however,  followup  discussions  with 
respondents  revealed  misconceptions  about  design  activities.  An  example:  Persons  involved  in  teaching  long  term 
coureses  mistook  normal  course  maintenance,  the  process  of  adding  new  information  to  a course  and  deleting  outdated 
information,  with  formative  evaluation. 

No  respondent  stated  they  completed  all  the  design  activities  100%  of  the  time,  and  only  one  respondent  stated 
they  “usually  + always”  completed  design  activities. 

Of  the  activities  most  frequently  accomplished  writing  learning  objectives  was  moot  often  completed,  followed 
by  piloting  test  instructions.  Writing  learning  objectives  was  the  only  activity  identified  as  being  always  done  by  more 
than  half  of  the  respondents. 

Why  activities  get  excluded 


Table  3 Reasons  for  exclusing  ID  activity 


Activity 

NA 

Lack 

expertise 

Client  non- 
support 

Decision 

already 

made 

needs 

assessment 

6 

3 

5 

21 

Training  as 
solution 

5 

2 

23 

Conduct  a task 
analysis 

4 

5 

4 

12 

Write  learning 
objectives 

7 

4 

1 

5 

Identify 

learning 

outcomes 

5 

6 

1 

Assess  entry 
level  skills 

2 

3 

2 

8 

Develop  test 
items 

6 

3 

1 

4 

Select 

instructional 

strategies 

4 

5 

1 

11 

Pilot  test 
instructions 

8 

3 

1 

Conduct 
formative 
1 evaluation 

5 

1 

1 

3 

2 

O 

ERIC 
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Table  3:  continued 


Activity 

Unnessary 

No  time 

No 

money 

Travel 

restrictions 

process 
not  in 
place 

needs 

assessment 

4 

9 

4 

2 

5 

Training  as 
solution 

1 

1 

2 

1 

task 

analysis 

6 

2 

1 

2 

Learning 

objectives 

1 

1 

1 

1 

Learning 

outcomes 

1 

4 

Assess  entry 
level  skills 

3 

7 

2 

4 

7 

Develop  test 
items 

4 

2 

3 

Select 

instructional 

strategies 

2 

2 

select  media 
formats 

1 

1 

3 

pilot  test 
instructions 

1 

7 

1 

2 

2 

conduct 

formative 

evaluations 

10 

6 

2 

8 

The  client  non-support  category  was  excluded  fromt  the  above  table  because  it  was  not  regarded  more  than  three 
timnes  as  a limitation  on  design  activities. 

The  most  common  reason  for  the  exclusion  of  design  activities  was  the  decision  had  already  been  made  prior  to 
the  designer  getting  the  project.  The  categories  where  this  reason  is  most  evident  are  conducting  needs  assessment, 
identifying  training  as  the  solution  to  the  instructional  problem,  and  conducting  a task  analysis. 

Discussion 

The  results  of  the  survey  indicate  that  not  all  design  activities  are  done  for  all  projects.  In  the  above  cases,  the 
reasons  for  the  decision  already  being  made  can  be  loosely  grouped  into  the  category  of  “corporate  culture.”  Decisions 
about  what  projects  were  to  be  taken  on  as  instructional  projects  were  generated  largely  through  a corporate  Program 
Management  Process(PMP)  which  has  as  a requirement  a preliminary  education  plan  prior  to  the  product’s  going  through 
a gated  process  (in  this  case  an  idea  gate.)  The  preliminary  plan  is  supposed  to  be  generated  by  a responsible  manager. 

No  manager  involved  with  this  study  took  a formal  part  in  any  design  team.  Only  one  filled  in  the 
questionnaire  but  qualified  the  answers  by  saying,  “from  the  time  I was  an  education  specilaist.”  Other  management 
people  responded  by  stating  the  questionnaire  was  not  germane  to  them.  One  can  conclude  from  the  survey  that 
determination  of  activities  to  include  for  any  given  project  are  largely  predetermined. 

Partly  the  above  conclusion  derives  from  lack  of  knowledge  on  the  part  ot  both  management  and  employee  of 
ISD.  Historically,  STK  has  approached  instructional  r*  mlerns  from  the  same  base,  i.e.  STK  has  always  implemented 
new  product  training  so  a lecture/lab  course  is  an  expectation  of  the  corporation  for  any  new  product.  There  is  both  a 
knowledge  gap  and  cultural  impediment  to  following  the  ISD  process  at  STK. 


0 


462475 


Conclusions 


Steps  are  ommitted  from  ISD  because  they  either  are  predetermined  (decision  already  made)  or  they  are  largely 
misunderstood  by  those  trying  to  implement  them.  As  in  Wedman  and  Tessmers’  (1993),  ISD  practise  is  a situational 
activity  at  STK.  The  value  of  the  ISD  model  must  also  be  questioned.  Clearly  the  corporation  doesn’t  hire  personnel 
based  on  their  education  in  ISD,  but  rather  for  their  skill  in  a specific  content  area.  Knowledge  of  ISD  is  a 
supplementary  skill. 

The  result  of  the  study  points  to  the  conclusion  that  traditional  ISD  models,  after  which  STK  processes  are 
patterned,  are  not  followed,  and  therefore  not  of  value  to  the  corporation  other  than  to  satisfy  a need  to  implement  a 
standard  for  doing  business.  STK  doesn’t  generally  hire  people  with  that  knowledge,  therefore  the  model  should  be 
discarded  at  STK  and  replaced  with  a more  useable  way  of  producing  timely  and  effective  instruction  or  instructional 
alternatives  when  called  for. 

Implications  for  practicing  Instructional  Designers 

A high  tech  corporation  such  as  STK  values  subject  area  knowledge  more  than  ISD  knowledge.  This  points  to 
a possible  dichotomy  in  the  future  of  the  field.  Since  the  corporation  is  reluctant  to  hire  people  with  strictly  ISD 
background,  such  persons  might  be  better  suited  to  consulting.  What  STK  values  is  content  knowledge.  Engineering 
managers  are  reluctant  to  offer  engineers  as  subject  matter  experts,  therfore  engineering  knowledge  becomes  a valued 
commodity. 

Areas  for  furths.r.study. 

► The  question  of  whether  STK  follows  a typical  pattern  for  technical  corporations  might  provide  a fertile  area  for  further 
research.  Anecdotal  information  on  two  like  corporations,  HP  and  Sun  indicates  their  technical  training  departments 
perform  other  functions,  such  as  consulting  and  product  support.  Furthermore  they  often  outsource  soft  skill 
components  of  training. 
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Problem 

MIDI  keyboards  are  being  introduced  in  school  curricula.  Their  effects  beyond  music  instruction  are  not  known. 

Questions 

a.  Does  school  instruction  in  MIDI  keyboards  improve  memory  skills? 

b.  Does  school  instruction  in  MIDI  keyboards  improve  sentiment  toward  school? 

Participants 

Pupils  in  early  elementary  grade  at  five  schools  were  evaluated.  The  pupils  and  the  schools  differed  demographically. 
School  1 was  in  a poor,  urban  center.  All  the  pupils  were  black  African-Americans.  School  2 was  in  a different  poor 

urban  center.  The  group  of  pupils  was  nearly  evenly  split  racially  between  black  and  white.  The  third  school  was  in  a 

middle-class  suburb.  Nearly  all  of  the  pupils  were  white.  The  fourth  and  fifth  schools  were  in  the  same  large  semi-rural 
school  district.  All  of  the  children  were  black.  The  latter  two  schools  were  compared  and  those  results  are  reported. 

Variables 

Memory  skills  and  sentiment  towards  school  were  measured  since  these  variables  represented  areas  that  were 
fundamental  to  learning  and  were  therefore  relevant  for  cross-curricular  education  and  because  of  the  likelihood  that 
keyboard  instruction  could  contribute  to  their  development. 

Instruments 

Two  assessment  instruments  were  used.  Subtests  of  the  Wide  Range  Assessment  of  Memory  Learning  (WR  AML) 
by  Sheslow  and  Adams  (1990)  measured  memory  skills.  A School  Sentiment  Inventory  was  a survey  developed  to  gauge 
sentiment  towards  school  and  instructional  media  including  the  keyboard. 

Procedures 

Pupils  were  interviewed  at  the  beginning  and  end  of  a school  year  for  both  measures.  During  the  year  comparison 
groups  were  provided  regularly-scheduled  music  instruction  for  one  half-hour  per  week  with  the  same  instructor. 

Analysis 

Interview  scores  were  compared  for  each  group  for  the  beginning  and  end  of  the  school  year.  Scores  were  also 
compared  between  groups  for  both  periods. 


Results 

There  were  no  discernible  gains  in  memory  skills  that  wero  clearly  attributable  to  keyboard  instruction. 
Overall,  the  various  analyses  of  school  sentiment  data  favorably  support  the  groups  that  conducted  music 
instruction  with  keyboards. 
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Discussion 


The  School  Sentiment  Inventory  results  were  encouraging  regarding  the  integration  of  piano  keyboards  into  an 
elementary  school  curriculum.  We  surmise  that  the  pupils*  achieving  competence  in  keyboard  instruction  contributed  to 
their  approval  ratings  of  school  and  music  instruction.  The  sentiment  underlying  the  ratings  may  develop  a value  for 
keyboard  instruction  that  may  transfer  to  other  learning  or  school  work. 

The  lack  of  differences  in  terms  of  affecting  memory  skills  can  perhaps  be  explained  by  the  fact  that  the  pupils  were 
engaged  in  about  thirty  minutes  of  music  instruction  per  week.  It  is  unlikely  that  this  was  enough  time  to  develop 
memory  skills  that  were  observable  beyond  music  performance.  At  the  same  time,  the  favorable  sentiment  towards  music 
and  school  can  be  attributed  to  this  short  amount  of  instructional  time. 
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Abstract 


Four  studies  were  conducted,  two  with  independent  groups  of  practicing  teachers  and  two  conducted  longitudinally, 
once  with  a single  group  as  preservice  undergraduate  teachers,  and  again  subsequently  after  they  had  taught  professionally  for  a 
year.  A purpose  of  these  studies  was  to  identify  levels  of  computer  use  among  practicing  teachers  and  change  from  expected  to 
actual  levels  of  use  among  the  preservice-to-novice  group.  Another  purpose  was  to  identify  variables  of  internal  motivation  as 
predictors  of  teachers’  computer  use.  Levels  of  computer  use  among  teachers  were  not  high.  Preservice  students’  expectations 
of  computer  use  were  very  high  then  dropped  after  one  year  of  professional  teaching.  Perceived  relevance  of  and  self- 
competence  in  computer  use  were  predictors;  when  subjective  norms  was  included  in  the  calculation  it  emerged  as  the  sole 
predictor. 

Introduction 

Underutilization  of  computers  for  teaching  has  occurred  in  spite  of  two  prevailing  conditions  that  would  seem  to 
promote  their  use.  First,  there  is  much  support  for  the  opinion  that  educational  computer  technology  could  significantly 
improve  the  educational  system  (National  Task  Force  on  Educational  Technology,  1986;  Shanker,  1990;  Sheingold  & 
Hadley,  1990;  United  States  Office  of  Technology  Assessment  [OTA],  1988).  Secondly,  the  microintensity  level— ratio 
of  computers  per  learner  has  improved  from  1 12.4:1  in  1983  to  18.9:1  in  1992  (OTA,  1988;  Market  Data  Retrieval, 

1992).  Yet,  despite  increased  availability  and  support  for  computers,  relatively  few  teachers  have  integrated  them  into 
their  teaching.  A survey  of  teachers  who  were  exceptional  users  of  computers  for  teaching  averaged  only  about  one  such 
teacher  per  school  (Sheingold  & Hadley,  1990).  This  paucity  of  integration  occurred  even  though  the  availability  of 
computers  (59)  in  the  schools  surveyed  was  more  than  double  the  average  number  of  computers  (26)  available  for  schools 
in  the  United  States  (Becker,  1989).  Extraordinary  abundance  of  computers  was  not  matched  by  an  abundance  of 
extraordinary  users  of  computers.  The  result  of  this  imbalance  is  that  computers  are  underutilized. 

These  studies  were  undertaken  to  identify  what  teacher  variables  might  predict  computer  use.  First,  computer  use 
was  classified  as  a process  of  the  adoption  of  innovation  (Hall,  1982;  Rogers  1962,  1983;  Rogers  & Shoemaker,  1971) 
or  more  specifically  Instructional  Transformation  (Hooper  & Rieber,  1995;  Rieber  & Welliver,  1989;  Welliver,  1990). 

Secondly,  teachers’  computer  use  was  examined  for  the  influence  of  personal  variables  to  computer  use.  In  a 
broad  study  of  the  implementation  of  innovation  in  education,  this  class  of  variables  was  excluded  (Berman  & 

McLaughlin,  1978).  Yet,  information  about  personal  variables  may  be  the  most  valuable  for  influencing  behavior  or 
performance  (Coovert  & Goldstein,  1980;  Gallo,  1986;  Jorde-Bloom  & Ford,  1988).  Sheingold  and  Hadley’s  survey 
(1990)  suggested  that  teachers  who  were  exceptional  users  of  computers  for  teaching  were  also  highly  motivated.  Because 
of  the  prominence  of  motivation  as  a personal  variable  relative  to  computer  use,  it  was  the  focus  of  the  studies.  From 
within  the  broad  construct  of  motivation,  Expectancy  Theory  (Vroom,  1964)  and  subjective  norms  guided  the  selection  of 
variables  (Ajzen  & Fishbein,  1980;  Fishbein,  Jaccard,  Davidson,  Ajzen,  & Loken  1980) 

Problem 

Expectations  for  teachers  to  integrate  educational  computing  into  teaching  are  high.  Levels  of  integration  among 
teachers  are  low.  Educational  computing  is  underutilized. 

Questions 

1 . What  are  the  levels  of  computer  use  among  teachers? 

2.  What  are  the  changes  from  student- teachers  expectations  of  computer  use  after  one  year  of  professional  teaching? 

3.  What  internal  motivating  variables  predict  computer  use  by  teachers? 

Participants/4  Studies 

Four  studies  were  conducted:  two  independent  groups  of  practicing  teachers  and  group  studied  twice  longitudinally. 
The  latter  group  first  participated  as  undergraduate  preservice  teachers.  Then,  they  participated  after  one  year  of  professional 
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aching.  Participants  with  an  elementary  school  focus  were  selected  to  control  for  the  prevalence  of  computer  use  in  certain 
subjects  as  might  be  the  case  with  secondary  or  post-secondary  teachers. 

Longitudinal  Studies 

a.  Preservice  undergraduates  ( n = 167) 

b.  Novice  teachers  (same  as  group  ‘a’  but  after  1 year  of  teaching)  (n  = 100) 

Independent  Studies 

a.  Practicing  teachers  ( n = 170) 

b.  Practicing  teachers  ( n = 138) 


Variables 

Criterion 

• Levels  of  Computer  Use 
Independent 

• Subjective  norms  (only  for  the  last  study) 

• perceived  relevance  of  computers 

• self-competence  in  using  computers 

• innovativeness 

• teacher  locus  of  control 

Demographics 

•age 

• gender 

• computer  experience 


Instruments 

a.  Levels  of  Computer  Use  scale  (LCU)  (Marcinkiewicz  & Welliver,  1993) 

b.  subjective  norms  scale  (Fishbein,  Jaccard,  Davidson,  Ajzen,  I.,  & Loken,  1980;  MacDonald,  1991;  Marcinkiewicz, 
1994) 

c.  survey  for  perceived  relevance  and  self-competence 

d.  Innovativeness  Scale  (Hurt,  Joseph,  & Cook,  1977) 

e.  Teacher  Locus  of  Control  (Rose  & Medway,  1981,  1982);  Teacher  Role  Survey  (Maes  & Anderson,  1985) 

Procedures 

Participants  completed  questionnaires  composed  of  the  above  measures.  The  instructions  for  the  questionnaire  for  the 
preservice  undergraduates  were  worded  to  elicit  their  expectations.  The  preservice  sample  was  also  split  in  its  use  of  the  two 
locus  of  control  measures.  Scores  were  then  standardized  as  T scores. 

Analysis 

Levels  of  use  were  scored  with  the  LCU  scale.  Exploratory  intercorrelations  were  calculated.  Logistic  regressions  were 
calculated  to  determine  the  predictor  variables. 


Results 

For  all  studies,  perceived  relevance,  self-competence,  and  innovativeness  were  correlated;  however,  for  the  study  with 
the  novice  teachers  the  correlation  between  self-competence  and  perceived  relevance  was  double  that  of  previously  scored. 

The  variables  that  were  identified  as  predictors  are  listed  in  Table  2.  The  chronological  order  in  which  the  studies 
were  conducted  is  listed  in  Table  2 from  left  to  right.  The  only  variable  in  the  last  study  conducted  that  was  identified  as 
contributing  to  the  prediction  of  teachers1  computer  use  was  subjective  norms. 
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Table  1 


Levels  of  Computer  Use 


Practicing  1 Practicing  2 Avg.  % of 

Practicing 

Pre  service 

Novice 

Computer  Use 

Nonuse 

43.5 

31.0 

37.2 

2.7 

39 

Utilization 

48.5 

66.0 

57.2 

84.0 

60 

Integration 

8.0 

3.0 

5.5 

13.3 

1 

Table  2 

Predictors  of  Computer  Use 

Practicing  1 

Preservice 

Novice 

Practicing  2 

self-competence 

p < .000 

p < .04 

N 

N 

perceived  relevance 

N 

p < .005 

N 

N 

innovativeness 

p < .24 

N 

N 

N 

teacher  locus  of  control 

N 

N 

p < .03* 

N 

subjective  norms 

X 

X 

X 

p < .01 

* Teacher  Locus  of  Control  for  Preservice  predicted  Novices  use. 
X Subjective  norms  was  not  inc'uded  in  these  studies. 

N No  Significant  Difference 


Discussion 


In  comparison  with  the  expectations  of  the  preservice  teachers,  the  levels  of  computer  use  of  among  practicing 
teachers  is  low.  However,  while  the  preservice  teachers  had  very  high  expectations  of  computer  use,  after  they  had  taught  for  a 
year  their  levels  of  use  approached  the  average  levels  of  use  of  the  practicing  teachers.  This  may  be  attributable  to  regression 
to  the  mean  behavior,  ambitious  undergraduate  expectations,  or  the  effects  of  an  excellent  undergraduate  program.  Still,  the 
differences  in  use  between  the  two  groups  were  statistically  significant. 

For  all  the  studies  except  for  the  Novice  round  of  the  longitudinal  one,  there  is  a pattern  from  among  the  personal 
variables  that  predict  computer  use.  Self-competence  and  perceived  relevance  are  highly  correlated.  This  correlation  is  most 
dramatic  between  rounds  1 and  2 of  the  longitudinal  study  when  the  correlations  nearly  double  in  strength.  Self-competence 
emerges  as  a predictor  more  often  than  do  perceived  relevance  and  innovativeness  but  since  they  are  also  correlated  in  different 
studies,  it  may  be  that  together  they  are  indicators  of  motivation  to  computer  use.  The  necessity  of  feeling  competent  in  the 
use  of  computer  technology  is  intuitively  appealing  and  is  also  predicted  by  expectancy  theory.  The  increased  strength  of 
correlation  between  self-competence  and  perceived  relevance  during  the  Novice  Study  suggests  the  importance  ot  these 
variables;  their  failure  to  predict  computer  use  may  be  due  to  the  increase  in  nonuse  since  the  first  round.  Perhaps  most 
compelling  is  the  emergence  of  subjective  norms  as  a predictor  superior  to  either  of  the  previously  identified  ones.  While 
subjective  norms  reflects  one’s  personal  motivation  it  also  reflects  motivation  based  on  one’s  perceptions  of  others’ 
expectations — akin  to  phenomena  such  as  peer  pressure,  introjccted  regulation,  or  values  building.  This  result  is  consistent 
with  the  view  that  the  expectations  of  a teacher’s  culture  as  embodied  in  one’s  administrators,  colleagues,  students  and 
professional  societies  significantly  influence  a teacher  to  use  computers. 
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Availability 


The  availability  of  computers  is  sometimes  suggested  as  a factor  in  determining  teachers  use.  These  studies  focused  on 
personal  motivation.  It  is  important  to  consider  availability  and  motivation  as  complementary  and  not  as  mutually  exclusive. 
A model  for  competent  human  behavior  was  suggested  by  Gilbert  (1978)  in  which  the  three  dimensions  of  information, 
motivation,  and  equipment  are  crossed  with  external  and  internal  perspectives.  Gilbert’s  model  provides  a context  which 
encompasses  the  need  for  the  availability  of  computer  equipment;  it  also  acknowledges  motivation.  In  sum,  teachers  will  not 
adopt  computers  unless  all  dimensions  are  satisfied.  When  considering  the  dimension  of  motivation,  subjective  norms  is 
predictive  of  teachers  computer  use. 


Implications^ 

In  order  for  teachers  to  adopt  computers,  there  needs  to  be  a perception  generated  by  the  professional  environment 

that  computer  integration  is  expected.  This  can  be  established  by  modeling  use  by  administrators,  colleagues,  students,  and 

the  profession.  A work  environment  would  be  equipped  and  faculty  training  and  support  would  also  be  available. 
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Introduction 

Communication  technologies  now  exist  to  present  live  interactive  instruction  in  real  time  via  distance  learning. 
Video  Teletraining  (VTT),  also  known  as  interactive  television,  is  an  application  of  distance  education  presented  by  a 
two-way  audio  and  two-way  video  system.  The  major  advantage  of  VTT  is  that  it  offers  live  instruction.  While  other 
technologies  may  be  both  viable  and  cost  effective,  the  option  of  live  instruction  available  through  VTT  may  outweigh 
other  distance  learning  strategies.  Using  a myriad  of  media- print,  audio,  video,  computers,  supplemental  technologies 
(e.g.,  the  telephone  and  facsimile  machine),  and  off-line  equipment  (e.g.,  videotape  players)-provides  instructors  and 
students  with  a means  for  communicating  and  learning  that  may  be  almost  as  good  as  being  there. 

The  Florida  Teletraining  Project  (FTP)  was  funded  by  the  Department  of  Defense  to  test  the  feasibility  of  using 
a video  teletraining  network  (VTT)  (two-way  audio/two  way  compressed  video)  to  present  military  instruction  to 
reservists  in  Florida.  This  program  was  to  be  conducted  by  two-year  community  colleges  in  collaboration  with  armed 
forces  schools,  in  part  to  test  the  capability  of  the  community  colleges  to  provide  VT l'  instruction  to  reservists. 

Objectives 

The  FTP  was  directed  to  identify,  collect,  and  evaluate  telecommunications  and  pilot  project  test  data.  The 
specific  project  objectives  were  to: 

ascertain  the  merit  of  using  telecommunications  training  provided  by  non-military  sources 
(i.e.,  community  colleges)  for  training  military  personnel 

quantify  the  value  of  the  instruction  received 

guide  future  government  and  DoD  decisions  related  to  distance  learning. 

Twenty-one  evaluation  objectives  were  identified  for  the  project  in  the  following  categories:  technology, 
instruction,  community  colleges,  and  cost.  Data  related  to  each  of  the  objectives  was  collected  from:  (a)  students,  (b) 
course  developers  who  were  also  the  VTT  instructors,  (c)  all  remote  site  personnel  including  Instructional  Coordinators 
(ICs),  Points  of  Contact  (POCs),  and  technicians,  (d)  administrative  project  staff  at  the  origination  site,  (e)  technical  and 
production  teams,  and  (f)  military  personnel  including  subject  matter  experts  (SMEs),  on-camera  military  instructors,  and 
military  remote  site  personnel. 

These  evaluation  objectives  focused  on  the: 

technology  selected  for  the  project  (e.g.,  ascertaining  its  reliability,  the  target  community’s 
acceptance  of  it,  and  how  appropriate  it  was  for  providing  the  training) 

instruction  (e.g.,  determining  whether  or  not  the  students  met  the  learning  objectives,  and  the 
effectiveness  of  the  instructional  strategies) 

community  college  (e.g.,  roles  and  responsibilities  as  providers  of  military  instruction) 

costs  (e.g.,  developing  a cost  model,  determining  the  cost  of  course  design  and 
implementation,  estimating  the  cost  of  an  operational  system,  and  comparing  the  cost  of  a 
telecommunications  approach  to  the  cost  of  selected  conventional  training  options). 

Video  Teletraining  in  the  Military 

Dwindling  resources  and  the  size  and  importance  of  the  reserve  component  are  two  of  the  main  reasons  the  U S. 
military  has  become  interested  in  distance  education.  Specifically,  reservist  must  be  trained  to  the  same  standard  as  the 
active  forces  but  they  cannot  be  trained  on  a daily  'asis.  This  need  for  increased  training  has  prompted  the  military 
services,  industry,  and  academia  to  conduct  research  on  the  feasibility  of  training  in  a distance  mode.  While  the  research 
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is  in  its  infancy  and  many  of  the  studies  have  limitations,  researchers  have  drawn  two  primary  conclusions:  (a)  students 
typically  do  as  well  on  learning  outcomes  using  distance  education  methods  as  they  do  when  taught  by  conventional 
methods,  and  (b)  student  satisfaction  using  distance  education  is  equal  to  or  higher  than  classroom  instruction 
(DeLoughry,  1988;  Fahl,  1983;  Grimes,  Neilsen,  & Niss,  1988;  Keene  & Cary,  1990;  Kruh,  1983;  Partin  & Atkins, 
1984). 


The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC  )(1987;  1990;  1991)  supports  the  use  of  VTT  and 
maintains  that  it  is  both  an  effective  and  cost  efficient  method  for  presenting  instruction  to  RC  forces.  The  Naval  Air 
Warfare  Center,  Training  Systems  Division  (NAWCTSD)  has  also  conducted  a series  of  studies  that  addressed  different 
aspects  of  VTT.  They  too  have  concluded  that  VTT  was  a viable  alternative  for  providing  high  quality  and  cost  effective 
instruction  to  distance  learners. 

The  need  to  find  cost  effective  and  efficient  learning  strategies  has  led  to  conclusions  about  what  technologies  are 
viable  and  effective,  what  courses  should  be  selected  for  distance  education,  and  what  variables  are  key  when  designing 
distance  learning  courses.  The  NAWCTSD  developed  a course  selection  model  for  Navy  VTT.  The  major 
recommendations  from  this  study  were:  (a)  select  courses  that  have  a high  potential  for  savings,  e.g.,  courses  with  high 
throughput  and  short  duration,  (b)  select  courses  that  have  an  appropriate  mix  of  lecture  and  laboratory,  (c)  do  not  select 
courses  that  are  equipment  intensive,  and  (d)  do  not  select  courses  that  require  substantial  curriculum  modification. 

Course  Design  Considerations 

Studies  conducted  by  the  Army  and  the  Navy  have  concluded  that  distance  education  courses  typically  require 
more  extensive  planning  than  platform  instruction.  Haarland  and  Newby  (1984)  state  that  the  increases  in  student 
performance  and  satisfaction  may  be  due  to  improved  course  design  and  teaching  performance  rather  than  as  a function  of 
a specific  technology.  TRADOC  endorses  the  SAT  model  for  the  design  of  courses  and  states  that  effective  course 
delivery  must  take  into  account  proper  management  of  the  design,  resourcing,  development,  production,  distribution,  and 
evaluation  of  VTT  programs. 

At  the  core  of  interactive  television  is  the  concept  of  interactivity  (Moore,  1989;  Ritchie  & Newby,  1989; 
Stoffel,  1987).  Interactive  television  is  defined  by  the  fact  that  good  instruction,  whether  it  is  presented  in  a classroom  or 
at  a distance,  stresses  interaction  among  the  participants  in  the  teaching-learning  process. 

In  addition  to  interactivity,  the  following  course  design  features  are  also  necessary  for  successful  VTT  (Bailey,  et 
al.,  1989;  Cyrs  and  Smith,  1990;  Defense  Language  Institute,  1992;  McDonald,  et  al.,  1990;  Martin,  1993; 

Ostendorf,  1991;  Sheppard,  et  al.,  1990): 

group  dynamics  should  be  addressed 

student  involvement  activities  need  to  be  carefully  structured 

lecture  segments  should  not  exceed  20  minutes 

visual  aids  must  be  adapted  for  television  viewing 

careful  planning  is  required  to  handle  student  questions  and  discussions 

instructors  must  involve  and  motivate  learners 

an  interactive  study  guide  (ISG)  must  be  provided 

Because  military  projects  are  often  large,  management  issues  when  using  VTT  are  of  utmost  importance. 

Maloy  and  Perry  (1991)  addressed  the  policy  and  management  issues  of  a large  Navy  project.  They  found  that  a large 
teletraining  project  required  a team  approach.  Some  of  the  key  team  members  were  an  educational  specialist  (e.g.,  an 
instructional  designer  or  evaluation  expert),  an  engineer,  a budget  analyst,  an  audio-visual  specialist,  a security  specialist. 
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a researcher/analyst,  a resource  sponsor  specialist  (to  provide  support  at  the  highest  level),  and  representatives  from 
civilian  personnel,  fleets,  training  command,  and  reserves. 

In  summary,  effective  teletraining  courses  are  similar  to  other  effective  distance  education  programs.  The 
following  are  key  recommendations  for  successful  VTT  courses: 

Select  courses  that  are  cognitive  rather  than  psychomotor  and  that  have  high  demand. 

Course  design  and  development  is  a critical  component  of  a successful  VTT  course.  Typically 
VTT  courses  require  more  detailed  pre-planning  than  traditional  courses.  Time  and  resources 
must  be  allocated  to  the  pre-planning  stages. 

Courses  must  be  designed  to  facilitate  interaction,  to  provide  feedback,  to  be  motivating,  and 
to  humanize  the  instruction.  Course  developers  must  be  knowledgeable  in  the  principles  of 
learning  and  instruction. 

Instructional  personnel  are  one  of  the  most  important  aspects  of  a VTT  course.  They  may 
require  considerable  training  and  practice  to  be  effective. 

It  takes  a team  of  people  to  design  and  deliver  good  VTT  instruction.  This  team  must  be 
organized  and  the  roles  and  responsibilities  of  all  personnel  must  be  delineated. 

Equipment  failures  occur.  Contingency  plans  must  be  prepared.  These  plans  may  require  the 
expenditure  of  additional  time  and  resources. 

In  large  scale  projects,  all  the  various  management  and  organization  functions  need  to  be 
coordinated.  Someone  who  is  knowledgeable  in  all  aspects  of  VTT  needs  to  maintain  a big 
picture  perspective. 

Overview  of  the  Florida  Teletraining  Project 

Five  courses  were  reconfigured  for  delivery  on  the  U.S.  Army  Teletraining  Network,  TNET.  Three  U.S.  Army 
Reserve  Component  Configured  Courseware  (RC3)  Military  Occupational  Specialty  (MOS)  courses  and  two  Navy  special 
topics  courses  were  conducted.  The  MOS  courses  were  delivered  once  each  to  Army  National  Guard  and  Army  Reserve 
soldiers  who  were  seeking  to  be  reclassified  in  these  MOSs: 

Army:  Unit  Administrative  Specialist  (71L,  level  10— entry  level)  This  73-hour  course  was 
presented  in  a two  week  block.  The  71L10  soldier  is  responsible  for  routine  office  administration  and  works  at 
various  organizational  levels  throughout  the  Army.  Twenty-six  hours  of  on-site  typing  instruction  were 
provided. 

Army:  Unit  Supply  Specialist  (76Y,  level  10).  This  96-hour  phase  of  the  76Y10  course  was 
presented  in  a two-week  block.  The  76Y10  soldier  performs  unit  and  organizational  supply  tasks,  including 
receipt,  storage,  issue  and  accountability  of  supplies  and  equipment. 

Army:  Basic  Military  Police  (95B,  level  10).  This  66-hour  phase  of  the  95B10  course  was  presented 
at  a two-week  block.  The  entry  level  MP  performs  the  tasks  of  apprehension  and  search,  patrol  and  traffic 
operations,  investigations,  physical  security,  and  self-defense. 

The  two  special  topics  courses  addressed  joint  services  needs  and  were  made  available  to  members  of  interested 
services  and  components: 

Navy:  Handling  Hazardous  Waste  (HazWaste).  This  was  offered  three  times  as  a one-day  course. 
Course  topics  included  a review  of  pertinent  laws  and  regulations,  a discussion  of  the  physical  and  chemical 
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properties  of  hazardous  materials,  the  correct  techniques  for  delivery  and  transfer  of  materials  to  hazardous  waste 

collection  sites,  and  pollution  and  spill  prevention. 

Navy:  Total  Quality  Leadership  (TQL).  This  was  offered  two  times  as  a one-day  workshop.  It  was 

an  introduction  to  the  Navy’s  adaptation  of  W.  Edwards  Deming's  approach  to  continuous  quality  improvement. 

The  courses  were  delivered  to  three  Florida  community  college  remote  sites:  St.  Petersburg  Junior  College 
(SPJC),  Valencia  Community  College  (VCC)  in  Orlando,  and  at  FCCJ.  HazWaste  and  TQL  were  also  offered  at  two 
out-of-state  sites  during  the  final  administrations  of  these  courses:  Ft  Taylor  Hardin  in  Montgomery,  Alabama  and  Camp 
Fogarty  in  East  Greenwich,  Rhode  Island. 

Course  Reconfiguration 

The  five  courses  had  to  be  converted  from  the  standard  mode  of  platform  delivery  to  VTT  delivery.  The  five-  / 

component  Systems  Approach  to  Training  (SAT)  model  was  adapted  for  use  in  reconfiguring  the  courseware.  In  the 
adapted  model,  the  five  functions  of  the  SAT  were  included  (i.e.,  Analysis,  Design,  Development,  Implementation,  and 
Evaluation).  Due  to  the  complexity  of  the  reconfiguration  effort,  two  functions  were  added  (Revise  Instruction  and 
Management). 

The  five  courses  that  were  reconfigured  in  the  FTP  were  assigned  by  the  government.  Therefore,  the  traditional 
tasks  performed  during  the  analysis  phase  of  the  SAT  model  were  not  conducted  during  this  project.  However,  these 
courses  were  analyzed  for  their  suitability  for  VTT  according  to  the  criteria  provided  by  NAWCTSD. 

During  the  design  phase,  the  Programs  of  Instruction  (POIs)  and  syllabi  were  analyzed  to  determine  the  adequacy 
of  the  course  materials  from  an  instructional  design  perspective.  In  the  Development  Phase,  the  instruction  was 
developed  and  produced.  All  course  materials  were  developed  during  this  phase  of  reconfiguration.  Preparation  for,  and 
the  delivery  of,  instruction  were  the  primary  goals  of  the  implementation  phase  of  the  reconfiguration  process.  The 
Revise  Instruction  Phase  was  added  to  the  adapted  model  because  several  revision  and  validation  cycles  had  to  be 
performed  during  the  reconfiguration  process.  The  Management  Phase  was  also  added  to  the  adapted  SAT  model  because 
of  the  complexity  of  the  project.  Three  primary  groups  of  people  had  to  coordinate  multiple  tasks  for  the  success  of  the 
project:  (a)  the  design,  development,  production,  implementation,  and  evaluation  teams,  (b)  the  military  organizations 
and  groups  involved  in  the  project,  and  (c)  the  three  community  colleges. 

A considerable  amount  of  data  was  collected  during  delivery  of  the  courses  (Evaluation  Phase).  One  of  the  roles 
of  project  personnel  was  to  collect  data  from  students,  and  data  were  collected  from  all  project  personnel. 

Course  Delivery 

Instruction  was  presented  live  overTNET  (Compression  Laboratories’  Rembrandt  II  06™;  compression  rate 
256Kbps).  At  the  origination  site,  the  primary  person  responsible  for  content  presentation  was  the  VTT  Instructor.  The 
content  of  the  courses,  however,  required  that  a military  instructor/SME,  the  Military  Instructional  Assistant  (MIA), 
deliver  some  of  the  content.  During  delivery,  up  to  three  people  were  needed  to  implement  the  instruction. 

At  each  remote  site,  a community  college  site  coordinator,  known  as  the  Instructional  Coordinator  (IC),  was 
needed  as  the  instructor  of  record  for  each  course.  The  IC  also  performed  some  of  the  off-line  instructional  roles.  For  the 
MOS  courses,  a Military  Site  Coordinator  (MSC)  was  also  required.  The  IC  and  MSC  were  the  VTT  Instructors’ 
representatives  at  the  remote  sites.  It  was  their  responsibility  to  manage  the  instructional  activities  at  each  remote  site. 

Evaluation  Methodology 


Students 

Students  were  selected  by  their  respective  military  commands  for  participation  in  this  project.  A total  of  275 
students  were  trained  during  the  project.  All  four  services  (including  the  reserve  components  of  the  Army  and  the  Air 
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Force),  and  the  U.S.  Coast  Guard  were  involved:  Florida  Army  National  Guard,  Florida  Air  National  Guard,  U.S.  Army 
Reserve,  U.S.  Navy  (military  and  civilian  personnel),  U.S.  Marine  Corps,  U.S.  Coast  Guard,  Rhode  Island  Air  National 
Guard,  and  Alabama  Army  National  Guard. 

The  average  age  of  all  the  students  was  33.3 / years.  All  of  the  students  were  high  school  graduates  or  the 
equivalent  and  15%  had  a four-year  college  degree  or  more.  A military  grade  of  E5  was  selected  as  an  approximate 
estimate  of  the  numbers  of  students  who  were  managers  versus  those  who  were  first  line  supervisors  and  enlisted 
personnel.  Approximately  63  percent  of  the  students  were  E5s  or  below.  In  addition,  approximately  30  percent  of  the 
students  had  a military  duty  position  related  to  the  course  in  which  they  were  enrolled  and  4.9%  had  a civilian  occupation 
related  to  the  course  content 

Students  were  asked  how  interested  they  were  in  the  course  content  prior  to  taking  the  course.  Approximately 
73  percent  rated  themselves  as  very  interested  (5  on  a 5-point  scale).  Approximately  19  percent  reported  that  they  had 
taken  a course  taught  by  television 

Evaluation  Instruments 

There  were  40  different  data  gathering  instruments  developed  by  FTP  personnel  in  conjunction  with  Systems  for 
Training  and  Applied  Research,  Inc.,  Lexington,  Kentucky.  These  included  achievement/proficiency  test,  student  course 
perceptions,  student  ratings  of  course  components,  and  interview  forms.  The  achievement/proficiency  tests  were  criterion 
based.  Those  for  the  MOS  courses  were  Performance  tests  (PTs)  developed  by  the  military.  Other  performance  tests  were 
developed  by  project  personnel.  In  addition,  six  standard  Army  forms  were  also  used  to  collect  test  data. 

Typical  items  from  the  instruments  developed  by  the  FTP  include  the  following: 

Dichotomous  data,  usually  "yes/no"  responses  or  "like/did  not  like,”  for  example, 

If  you  had  the  opportunity  to  take  additional  military ■ teletraining  instruction  in  the  future . 
would  you  want  to?  yes  no 

Ratings  on  a 3-  or  5-point  Likert  scale,  where  the  highest  number  is  always  the  most  positive 
response,  for  example, 

Please  rate  the  aspects  of  this  course  on  a scale  from  l to  5 or  mark  NA  if  the  item  is  not 
applicable  to  this  course: 

5 = Excellent 
4 = Very  Good 
3 = Good 
2 = Below  Average 
l = poor 

NA  = Not  Applicaole 

L The  VTT  instructors  poise,  / 2 3 4 5 NA 

personality  and  enthusiasm. 

2.  The  VTT  instructors  delivery  of  I 2 3 4 5 NA 

information  over  the  network. 
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Ranking  a series  of  responses  (coded  so  the  highest  number  was  always  the  most  positive),  for 
example. 


If  you  could  pick  from  the  following  military  training  options  in  the  future , which  would  you 
prefer  as  a method  to  receive  training? 

Traditional  classroom  instruction  at  a military  school 

Video  teletraining  at  a community  college  site 

Correspondence  study 

Training  provided  locally  by  an  assigned  military  instructor 

Video  teletraining  received  at  your  armory/reserve  center 

Open-ended  questions,  for  example. 

If  you  could  have  changed  something  in  the  course,  what  would  it  be? 


Procedures 

The  five  courses  produced  by  the  FTP  were  delivered  between  October,  1992  and  February,  1993.  Data  were 
gathered  from  the  course  participants  as  well  as  from  the  staff  responsible  for  facilitating  the  instruction. 

All  student  and  instructional  personnel  interviews  were  conducted  by  the  project  evaluators  according  to  a 
predetermined  schedule.  Individual  student  interviews  were  conducted  by  the  project  evaluators  immediately  after  the 
students  completed  the  questionnaire(s). 

The  evaluation  and  cost  data  were  coded  as  necessary  and  entered  into  a database.  The  Windows  version  of  the 
Statistical  Package  for  Social  Sciences  (SPSS  for  Windows  5.0)  was  used  to  analyze  the  evaluation  data. 

Results 

The  following  list  is  a brief  overview  of  the  major  findings  related  to  the  FTP.  A more  complete  analysis  of 
the  data  can  be  obtained  in  Bramble  and  Martin  (1995),  Martin  and  Bramble  (in  press);  and  Martin,  B.  L,  et  al.  (1994): 

Technology: 


TNET  was  99%  reliable;  students  and  instructional  personnel  rated  the  quality  of  the 
system  high. 

Students  and  ICs  indicated  that  they  preferred  a VTT  approach  to  traditional  training 
at  a military  facility. 

The  courses  selected  for  VTT  presentation  were  generally  suitable  for  the  technology 
although  some  modifications  were  made  (e.g.,  off-line  activities)  to  accommodate  the 
technology  and  the  course  content. 

Even  though  training  was  provided,  the  instructional  personnel  felt  that  more  training 
and  practice  was  needed  on  TNET. 


Instruction 


The  most  important  objective  was  the  quality  of  instruction.  All  students  passed  the  stated 
learning  objectives,  and  over  90%  of  all  students  in  the  MOS  courses  passed  the  performance  tests  on 
the  first  attempt  (See  Table  1 ). 
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No  comparisons  could  be  made  between  the  performance  records  of  students  in  the  VTT 
courses  and  those  in  other  training  options. 

Students  rated  the  learning  methods  and  activities,  including  the  interactivity  provided  in  the 
VTT  courses,  to  be  effective  (See  Table  2).  The  opportunities  for  interaction  were  rated  the  highest, 
although  interaction  with  students  over  the  network  was  the  most  difficult  interaction  to  achieve.  Based 
on  data  collected  during  the  interviews,  students  said  these  learning  methods  stimulated  their  interest 
and  they  were  generally  motivated  by  the  VTT  instruction. 

No  student  demographic  variables  predicted  success  or  lack  of  it  for  military  students  and  hence 
one  can  expect  that  VTT  is  an  acceptable  approach  for  the  general  military  population. 

Students  rated  all  the. instructional  personnel,  VTT  instructors,  MIAs,  ICs,  and  MSCs,  as 
effective.  A question  was  whether  or  not  civilians  can  provide  quality  instruction  to  military  students 
due  to  their  lack  of  military  training.  Some  students  felt  that  a military  instructor  should  have 
presented  the  instruction,  however,  all  students  passed  the  learning  objectives  for  all  five  courses. 

Community  Colleges 

FCCJ  had  the  technical  and  instructional  capabilities  to  implement  VTT  instruction.  The 
technical  staff,  TV  studio,  and  production  facilities  (one  of  the  best  among  community  colleges  in 
Florida)  were  excellent. 

Community  college  faculty  are  professional  educators.  While  the  faculty  at  FCCJ  lacked 
instructional  design  and  military  training  expertise,  they  were  able  to  design  and  present  high  quality 
instruction  given  specific  training. 

Staff  training  was  presented  to  all  instructional  and  technical  personnel.  This  training  was 
judged  to  be  of  high  quality  by  the  staff  and  they  indicated  that  it  enabled  them  to  perform  their  roles 
and  responsibilities. 

The  course  developers/VTT  instructors  had  the  most  extensive  training  of  all  the  instructional 
personnel.  This  included  instruction  in  VTT  presentation,  instructional  design,  and  military  training. 

The  community  colleges  were  able  to  grant  academic  and  continuing  education  credit  for  all 
courses.  Although  not  all  students  were  interested  in  receiving  credit,  those  that  were,  indicated  that 
college  credit  was  a definite  advantage  to  receiving  military  training  at  a community  college. 

The  three  community  colleges  were  organized  to  work  together  through  subcontracts;  this 
proved  to  be  a very  effective  mechanism. 

There  was  a complex  web  of  community  college,  university,  government,  and  military 
organizations  that  had  to  work  together  to  implement  this  project.  Formal  contracts,  MOUs,  IPRs, 
and  a number  of  informal  agreements  enabled  the  groups  to  work  together  effectively. 


Costs: 


With  the  exception  of  the  HazWaste  course,  the  VTT  courses  in  the  FTP  were  more  expensive 
when  compared  to  resident  training.  The  HazWaste  course  was  cost  effective  because  there  were  a large 
number  of  students  (N=l  16)  in  the  course  and  because  of  travel  and  per  diem  savings. 

The  costs  associated  with  providing  college  credit  (in-state  and  out-of-state)  and  the  lease  of  the 
four  TNET  systems  greatly  increased  the  costs  of  the  VTT  courses. 
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The  VTT  courses  become  more  cost  effective  if:  (a)  the  same  course  is  presented  more  than 
once  (because  design  and  development  costs  are  paid  only  once),  and  (b)  different  courses  are  presented 
in  the  same  month,  (because  each  course  would  be  charged  only  for  the  days  the  course  is  implemented 
rather  than  paying  the  TNET  costs  for  the  entire  month). 

Lessons  Learned 

The  pilot  test  and  project  provided  several  lessons  about  designing  and  implementing  VTT  instruction.  While 
some  lessons  are  more  relevant  to  military  instruction,  others  are  important  for  planning  any  VTT  project: 

The  choice  of  technologies,  in  this  case  TNET,  influenced  the  design  of  instruction  because  of 
the  video  transmission  rate.  The  FTP  courses  are  not  directly  adaptable  to  other  VTT 
systems.  They  would  have  to  be  modified  to  be  used  effectively.  Adapting  the  courseware 
will  typically  be  needed  when  changing  delivery  methodologies. 

Origination  and  remote  site  technicians  are  invaluable  resources.  Origination  site  technicians 
should  be  knowledgeable  in  various  communication  technologies. 

Adequate  staff  training  is  necessary  for  successful  VTT  instruction.  Instructors  and  course 
developers  need  instruction  in  basic  ISD,  how  to  conduct  instruction  using  the  selected 
distance  delivery  system,  and  how  to  operate  the  equipment. 

VTT  must  be  considered  in  the  context  of  other  training  options.  Courses  should  be  selected 
for  VTT  delivery  that  are  short  (one  to  two  weeks  in  length),  are  primarily  cognitive,  and  have 
high  demand  or  throughput. 

While  the  TNET  equipment  is  not  difficult  to  use,  some  personnel  are  intimidated  by  any  new 
technology.  Care  must  be  taken  to  provide  sufficient  practice  and  technical  support  to  such 
personnel  when  the  technology  is  in  use. 

Military  programs  of  instruction  (POI)  and  syllabi  can  be  successfully  reconfigured  for  VTT 
instruction.  The  quality  of  the  POIs  and  syllabi  received  from  the  military  directly  effects  the 
time  and  the  costs  for  reconfiguring  military  courses  for  VTT. 

Careful  attention  must  be  paid  to  designing  activities  that  allow  students  to  interact  with  each 
other  on  the  network. 

It  took  longer  to  present  RC3  courses  by  VTT  than  to  present  platform  instruction  because  of 
planned  interaction  and  the  administration  of  on-site  PEs.  Typically,  the  overall  course  length 
will  increase  when  using  VTT  in  order  to  present  the  same  content. 

Students  who  were  more  highly  educated  or  held  a higher  military  grade  or  rank  were  more 
critical  than  others  of  the  interactive  study  guide  (word  picture  concept)  that  was  used  in  this 
project.  Other  ways  of  involving  these  students  in  the  instructional  process  needs  to  be 
explored. 

The  different  cultures  of  the  military  and  the  community  college,  in  addition  to  the  roles  and 
responsibilities  assigned  to  each  group,  require  coordination  by  individuals  who  can  link  the 
two  communities. 

Course  design,  development,  and  delivery  required  a team  approach  because  the  community 
college  faculty  who  taught  the  courses  did  not  have  sufficient  military  background. 
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The  larger  the  population  that  can  be  served  using  VTT,  the  greater  the  cost  savings.  On- 
going VTT  programs  will  be  more  cost  effective  than  one-shot  courses. 


Conclusions 

Video  teletraining,  like  other  forms  of  distance  education,  can  be  a viable  option  for  increasing  training  and 
education  possibilities  for  military  reservists  as  well  as  for  non-military  populations.  The  costs  of  such  programs 
depend  on  a number  of  factors  including  the  technology  used,  how  extensive  the  course  design  and/or  reconfiguration 
effort  is,  the  number  of  support  personnel  required,  how  long  the  training  program  is,  and  how  much  staff  training  is 
required.  The  benefits  that  may  accrue  from  quality  distance  education  programs  include  improved  instruction,  broader 
access  to  training,  delivering  instruction  in  a variety  of  settings,  and  less  travel  by  course  participants. 

It  is  important,  however,  to  keep  in  mind  that  good  distance  education  programs  are  highly  dependent  on  good 
planning  (ISD),  good  instruction  (presentation),  and  good  organization  and  management  (Cornell,  1995;  Martin  & 
Bramble,  in  press).  Failure  to  attend  to  these  important  aspects  of  a distance  program  will  likely  result  in  poor 
instruction  and  learning  just  as  platform  instruction  that  is  not  well  planned  and  well  managed  will  also  result  in  poor 
instruction  and  learning.  That  is,  good  instruction  is  good  instruction  regardless  of  whether  it  is  presented  in  a classroom 
or  at  a distance. 

Note:  The  Florida  Teletraining  Project  was  conducted  under  contract  number  N6 1339-85  from  the  Defense  Training  and 
Performance  Data  Center  (TPDC)  to  the  Institute  for  Simulation  and  training  at  the  University  of  Central  Florida.  With 
the  closing  of  TPDC  in  1992,  contract  management  was  transferred  to  the  Defense  Institute  for  Training  Resources 
Analysis  (DOITRA).  Dr.  Steven  Skiles  and  Mr.  Bill  West  served  as  DoD  Project  Managers. 
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Table  1 


Pretest  and  Posttest  Performance  Data 


Course 

Performance 
Test  Measure 

Pretest 

(SD) 

(N) 

Posttest 

(SD) 

(N) 

t- value 
(df) 

ADMIN 

Timed  Typing 

20.48(9.58) 

(33) 

32.42(9.58) 

(33) 

11.41* 

(32) 

Type  a 

Memorandum 
(#  errors) 

7.42(4.81) 

(33) 

1.85(1.54) 

(33) 

6.97* 

(32) 

SUPPL 

50-item 

Achievement 

Test 

59.95(16.63) 

(40) 

76.00(11.25) 

(40) 

8.57* 

(39) 

MP 

90-item  PTa 

71.81(10.22) 

(26) 

92.58(10.22) 

(26) 

13.56* 

(25) 

HAZ 

20-itcm  PT 

47.68(13.53) 

(HD 

78.23(10.95) 

(111) 

21.54* 

(HO) 

TQL 

20-item  PT 

No  pretest  given 

82.90(11.01) 
(48  )b 

NA 

*£<.001,  one-tailed. 
aPT  = Performance  Test 

bNote:  There  are  1 1 missing  scores  from  the  Rhode  Island  site. 
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Table  2 

Selected  Student  Ratings  of  Instructional  Methods, 
Interactivity,  and  Course  Characteristics 
(5  Point  Scale,  5 = Highest) 

(SD) 

(N) 


ITEM 

ADMIN 

SUPPLY 

MP 

HAZ 

TOL 

Word  graphics 

3.65  (1.14) 
(31) 

3.82  (1.23) 
(38) 

3.44(1.04) 

(25) 

4.33  (0.77) 
(HO) 

4.43  (0.65) 
(47) 

Value  of 

Practical  exercises 

4.18  (0.98) 
(33) 

3.44(1.19) 

(32) 

4.04  (0.89) 
(25) 

4.26  (0.82) 
(111) 

4.15  (0.81) 
(47) 

Instructional  games 

2.61  (1.45) 
(28) 

3.44(1.19) 

(32) 

3.46  (1.25) 
(24) 

NA 

NA 

Remediation  helpful 

4.13  (0.88) 
(31) 

3.68  (1.02) 
(40) 

4.05  (0.80) 
(21) 

NA 

NA 

Opportunities  to  ask 
questions 

4.09  (0.89) 
(32) 

4.03(1.19) 

(40) 

4.31  (0.84) 
(26) 

4.18  (0.89) 
(HI) 

4.21  (0.77) 
(48) 

Interaction  w / 
students  on-site 

4.62  (0.61) 
(32) 

4.47  (0.82) 
(40) 

4.58  (0.50) 
(26) 

4.33  (0.74) 
(HI) 

4.11  (0.76) 
(47) 

Interaction  w/st. 
on-network 

3.38  (1.48) 
(32) 

3.50(1.41) 

(40) 

3.08  (1.06) 
(26) 

N/A 

N/A 

Interaction  w/VTT 
instructor 

4.23  (0.88) 
(31) 

4.15  (1.08) 
(40) 

3.60(1.15) 

(25) 

4.33  (0.74) 
(HI) 

4.23  (0.73) 
(47) 

General  course 
organization 

3.00  (1.22) 
(32) 

2.95  (1.18) 
(40) 

3.38  (1.06) 
(26) 

3.84  (0.93) 
(111) 

4.06  (0.95) 
(48) 

Quality  of  lesson 
presentations 

3.44  (1.01) 
(32) 

3.28  (1.13) 
(40) 

3.19  (1.17) 
(26) 

3.95  (0.85) 
(HO) 

4.00  (0.83) 
(48) 

O 

ERLC 
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Table  2 (continued) 


Time  to  cover 
topics 

2.69  (1.12) 
(32) 

3.68  (1.02) 
(40) 

3.42  (1.33) 
(26) 

3.24  (1.22) 
(HI) 

3.13  (1.21) 
(48) 

Overall  course 
quality 

3.59  (0.98) 
(32) 

3.55  (1.18) 
(40) 

3.19  (1.10) 
(26) 

4.04  (0.92) 
(111) 

4.10  (0.81) 
(48) 

Pacing  of  course 
(3=about  right) 

3.48  (1.34) 
(31) 

2.68  (0.91) 
(34) 

2.04  (1.06) 
(25) 

2.94  (0.86) 
(HO) 

3.11  (0.87) 
(47) 

O 

ERIC 
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Objectives 

This  paper  will  focus  on  the  training  aspects  of  ISO  9000  Quality  registration.  The  objective  of  this  paper  is  to 
report  on  an  application  and  case  study  in  a global  organization  that  successfully  achieved  ISO  9000  Quality  registration. 
An  essential  element  in  the  certification  process  is  an  appropriate  training  system.  The  training  system  is  important  in 
two  ways.  First,  the  communication  of  the  ISO  requirements  to  all  levels  of  the  organization  is  essential  for  certification. 
Second,  the  training  function  is  important  for  performance  improvement  to  maintain  quality  standards  and  continually 
improve  the  quality  processes.  The  following  content  areas  will  be  covered: 

• ISO  9000  Overview 

• ISO  9000  Requirements 

• Training  Requirements 

• Case  Study 

• Role  of  Information  Technology 

• Reducing  Costs 

ISO  9000  Overview 

Most  organizations  produce  a product  or  service  that  is  intended  to  satisfy  a user’s  needs  or  requirements.  Such 
requirements  are  often  incorporated  in  specifications.  Quality  is  traditionally  measured  in  terms  of  conformance  to 
specifications.  Conformance  to  specifications  does  not  guarantee  that  a customer’s  requirements  are  met.  Quality  system 
standards  and  guidelines  were  developed  that  complement  relevant  product  or  service  requirements  given  in  the  technical 
specifications.  The  series  of  international  standards,  ISO  9000  to  ISO  9004,  embodies  a rationalization  of  the  many  and 
various  national  approaches  in  this  sphere  (ISO  Quality  Standards  Collection,  1991).  The  ISO  9000  standards  were 
adopted  in  1987  by  the  International  Organization  for  Standardization  (ISO).  The  ISO  9000  certification  demonstrates  the 
capability  of  a supplier  to  control  the  processes  that  determine  the  acceptability  of  the  product  or  service  being  supplied 
(Rabbitt  and  Bergh,  1994). 

ISO  9000's  basic  requirement  is  that  there  must  be  adequate  management  of  whatever  activities  are  involved  in 
creating  products  (Oskarsson  & Glass,  1996).  The  ISO  standard  defines  and  documents  management’s  role  in  putting 
quality  into  a product.  Product  quality  had  traditionally  been  placed  in  the  technology  camp  with  procedures  such  as 
measuring  product  defects.  ISO  9000  is  in  the  quality  management  camp  instead  of  the  technology  camp.  The  standard 
is  written  at  a policy  level  with  little  technical  advise  on  how  to  accomplish  what  it  requires.  The  methods  seem 
superficial  and  bureaucratic.  In  addition,  he  standard  was  defined  by  quality  people.  The  originators  knew  a lot  about 
quality,  but  little  about  the  discipline  to  which  it  is  applied.  This  issue  is  particularly  problematic  for  functions  like 
R&D  and  processes  like  software  development,  where  the  products  have  no  weight,  are  created  by  knowledge  workers  and 
require  no  raw  materials  to  produce.  The  standard  also  originated  in  the  manufacturing  world  in  which  performance  gaps 
in  quality  are  measured  concretely. 

The  basic  benefit  of  the  ISO  9000  standard  is  that  it  is  a system  used  to  measure  and  evaluate  an  organization’s 
quality  management  system.  Assuring  customer  satisfaction  and  gaining  competitive  advantage  are  the  two  main  reasons 
for  incorporating  the  ISO  standard  in  a company.  The  ISO  9000  system  uses  formal  audits  and  a registration  procedure  to 
assure  customers  around  the  world  that  a supplier  is  applying  a high  level  quality  system  in  producing  and  delivering  its 
goods  or  services.  These  standards  are  now  being  met  in  companies  of  over  70  countries  worldwide. 

ISO  9000  Requirements 

ISO  9000  is  a family  of  standards  and  guidelines.  There  are  different  standards  based  on  the  type  of  business  that 
is  registered.  The  document  ISO  9000- 1 is  a general  guideline  that  gives  background  information  about  the  family  of 
standards.  ISO  9001,  ISO  9002,  and  ISO  9003  are  the  standards  in  the  family,  containing  requirements  on  a supplier. 

ISO  9002  and  ISO  9003  are  subsets  of  ISO  9001 . ISO  9002  is  used  for  situations  in  which  there  is  no  design.  ISO 
90003  is  used  in  situations  in  which  there  is  neither  design  nor  production  (e.g.  retail).  Typically  for  both  hardware  and 
software  product  development,  ISO  9001  is  the  standard  to  use.  ISO  9004  is  a comprehensive  guideline  to  the  use  of  the 
ISO  9000  standards.  ISO  9004-3  is  a guideline  on  how  to  use  ISO  9001  for  development.  ISO  9004-2  is  a guideline  for 
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the  application  of  ISO  9001  to  the  supply  of  services.  It  can  be  of  interest  in  the  context  of  information  technology, 
since  computer  centers  and  other  suppliers  of  data  services  can  benefit  from  its  use. 

The  ISO  9000  standard  focuses  on  20  aspects  of  a quality  program  that  are  subject  to  a rigorous  audit  during  the 
certification  process.  Each  section  relates  to  a specific  aspect  of  satisfying  customers.  These  elements  should  be  viewed  in 
terms  of  how  they  relate  to  a customer's  expectations.  The  following  20  elements  are  contained  in  section  4 of  the  ISO 
9000  standard  document. 

1 . Management  responsibility 

2.  Quality  system 

3.  Contract  review 

4.  Design  control 

5.  Document  and  data  control 

6.  Purchasing 

7.  Control  of  customer  specified  product 

8.  Product  identification  and  tractability 

9.  Process  control 

10.  Inspection  and  testing 

1 1 . Control  of  inspection ; measuring  and  test  equipment 

12.  Inspection  and  test  status 

1 3.  Control  of  non  conforming  product 

14.  Corrective  and  preventive  action 

15.  Handling , storage . packaging,  preservation  and  delivery 

1 6.  Control  of  quality  records 

1 7.  Internal  quality  audits 

18.  Training 

19.  Sen* icing 

20.  Statistical  techniques 

The  ISO  9000  elements  must  be  viewed  as  a system.  A system  is  a group  of  interacting  items  that  form  a 
unified  relationship.  A quality  system  is  defined  by  the  organization's  management  and  is  documented  in  the 
organization's  quality  system  manual.  The  manual  defines  the  organization’s  processes  according  to  the  20  ISO  elements. 
In  general  terms  a quality  system  is  defined  as  follows: 

Quality  System:  The  organizational  structure  of  responsibilities,  procedures,  processes,  and  resources  for 
implementing  quality  management  (ISO  Quality  Standards  Collection,  1991). 

ISO  9000  does  not  specify  quality  assurance  standards.  Those  standards  are  defined  by  the  organization.  ISO 
9000  provides  a model  for  quality  assurance  that  can  be  adapted  by  the  organization  as  it  sees  fit.  The  model  can  only 
survive  as  an  operational  system  if  various  departments  in  an  organization  continuously  interact  to  monitor/audit  the 
system  and  its  effectiveness.  Audits  are  required  every  year  by  independent  outside  auditors.  In  essence,  ISO  9000  is  a 
quality  system  that  attempts  to  certify  to  customers  that  an  organization  will  do  what  it  says  it  will  do  in  terms  of 
determining  product  requirements,  product  development,  product  manufacture  and  product  service. 

T raining  Requirements 

The  standards  require  that  staff  are  trained  for  their  tasks.  They  further  specify  that  the  supplier  to  have  a 
procedure  to: 

• Identify  training  needs  for  each  staff  position. 

• Provide  such  training. 

• Keep  records  of  the  training  of  all  staff  members. 
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Oskarsson  and  Glass  (1996)  highlight  examples  of  the  kinds  of  training  an  auditor  would  expect  to  see  planned 
in  software  development: 

• New  programming  languages  and  tools. 

• Audit  training  for  internal  quality  auditors. 

• Project  management  training. 

The  auditor  does  not  judge  an  organization’s  means  of  training  and  education  but  that  the  training  is  documented 
and  sufficient.  The  criterion  for  sufficient  training  is  that  a person  is  capable  of  performing  his/her  work  to  a high 
enough  standard.  The  auditor  may  see  this  in  quality  review  records  and  project  success.  The  records  of  training  are 
essential  for  compliance  with  the  standard.  Examples  of  typical  training  nonconformances  are: 

• No  procedure  for  planning  of  training. 

• No  training  records. 

• An  employee  who  has  not  received  proper  training  for  his  or  her  task. 

The  training  procedures  should  start  by  describing  how  you  identify  training  needs.  This  is  done  through  the  use 
of  job  descriptions  or  profiles.  Job  descriptions  should  detail  the  minimum  qualifications  required  for  assuming  a 
position  or  performing  a work  function.  The  job  descriptions  are  required  as  a basis  for  establishing  a training  program. 
The  job  description  should  state  what  the  skill  or  education  requirements  are  for  a particular  job.  Documentation  also 
needs  to  exist  for  each  individual  indicating  whether  they  meet  the  job  requirements.  It  is  not  always  possible  for  every 
new  employee  hired  or  promoted  to  be  trained  in  every  aspect  of  a job  the  moment  they  begin.  In  these  situations,  a 
formal  plan  should  be  developed  indicating  that  training  will  be  completed  in  a reasonable  amount  of  time.  Randall 
(1995)  suggests  developing  a training  needs  matrix  showing  the  departments  in  an  organization  on  one  axis  and  the  ISO 
element  on  the  other  axis  with  checks  in  the  matrix  indicating  that  a particular  department  undergo  more  detailed  training 
in  a particular  ISO  element. 

Documenting  training  programs  is  essential  in  the  ISO  standard.  Undocumented  training  programs  in  which  a 
large  amount  of  information  is  passed  verbally  from  one  person  to  another  do  not  meet  the  intent  of  the  ISO  requirement. 
This  does  not  mean  that  all  training  must  be  the  result  of  formal  classroom  training  courses.  Many  companies  have 
formalized  "on-the-job"  training  and  apprenticeship  programs.  These  training  programs  must  be  documented  and 
formalized  to  the  extent  that  they  go  beyond  mere  tribal  knowledge.  Training  and  education  techniques  such  as  action 
learning  need  to  be  documented  as  well. 

If  an  employee  meets  the  requirements  of  a position  because  of  their  experience,  it  is  not  necessary  that  they 
participate  in  training  that  is  not  needed.  Their  experience  should  be  documented  in  their  training  records  indicating  that 
they  meet  those  specific  requirements  of  the  job. 

Training  records  must  document  that  stated  qualifications  have  been  met  as  "appropriate  education,  training 
and/or  experience,  as  required."  Documenting  the  above  can  be  accomplished  with: 

• Certificates,  degrees,  diplomas,  licenses,  etc. 

• Course  descriptions,  course  outlines,  together  with  instructor  identification 
and  qualifications  to  teach  the  assigned  courses  or  skills. 

• Letters,  memos,  etc.,  documenting  experience. 

Case  Study 

This  case  study  involves  a division  of  a telecommunications  company  that  designs,  manufactures  and  installs 
transmission  equipment  for  customers  in  Canada,  the  United  States,  Europe  and  Asia.  The  implementation  of  the  ISO 
quality  system  was  performed  in  conjunction  with  other  quality  initiatives  including  statistical  process  control, 
continuo  improvement,  and  process  maturity/improvements.  T1  re  was  initial  senior  management  support  for 
pursuing  me  ISO  Quality  initiative,  both  at  the  corporate  and  divisional  levels. 
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' As  with  most  change,  the  ISO  9001  implementation  met  with  some  resistance  on  the  part  of  managers  and 
employees.  Resistance  originated  from  a lack  of  understanding  of  the  intent  of  the  ISO  quality  management  system,  of 
information  regarding  the  certification  process,  and  the  objectives  sought.  Some  managers  questioned  the  necessity  of  the 
ISO  Standard,  seeing  it  as  an  increase  in  paperwork.  Others  were  apprehensive  of  the  overtime  work  that  such  an 
implementation  would  require.  These  feelings  were  compounded  by  the  variety  of  quality  initiatives  that  had  come  and 
gone  throughout  the  company. 

The  ISO  system  was  implemented  in  phases  throughout  the  division.  First,  the  manufacturing  location  went 
through  the  ISO  9001  certification  program  and  achieved  certification  with  no  nonconformances.  The  success  of  this 
initial  certification  was  due  to  two  factors;  a small  core  team  of  less  than  20  people  who  took  responsibility  of  the 
elements,  and  a good  project  manager  who  was  able  to  bring  together  many  organizational  departments  for  a common 
purpose.  In  addition,  the  manufacturing  location  had  a strong  plant  manager  who  was  very  supportive  of  the  program. 
Certification  at  this  point  was  focused  on  the  local  level.  The  success  of  this  initial  effort  was  a good  learning  experience 
in  both  the  process  and  in  pulling  together  as  a team,  the  functional  units  of  the  organization  . 

The  rest  of  this  case  focuses  primarily  on  the  certification  for  the  R&D  part  of  the  division.  This  aspect  oi  the 
project  involved  many  international  elements  because  the  R&D  part  of  the  division  contained  work  groups  located 
throughout  the  US,  Can.ida,  and  the  United  Kingdom.  In  addition,  the  company  had  developed  several  improved  methods 
of  information  and  communications  technology  that  were  essential  in  implementing  certification  on  a global  scale. 

The  first  few  months  of  the  project  were  a sensitization  period  that  was  necessarv  to  convince  managers, 
especially  senior  managers,  of  the  advantages  of  the  ISO  9001  quality  system.  This  period  was  characterized  by  the 
personal  commitment  of  the  division's  Vice  President,  Assistant  Vice  Presidents  / Lab  Directors  at  each  lab.  Once  this 
commitment  was  achieved,  a core  team  was  formed.  At  each  R&D  location,  team  members  were  assigned  responsibility 
for  one  or  more  of  the  20  applicable  ISO  elements.  In  addition  there  was  an  executive  team  that  spanned  the  corporate 
and  divisional  locations.  Once  the  project  was  underway  weekly  video  meetings  as  well  as  conference  calls  were 
established.  The  division  also  utilized  a fairly  extensive  internal  computer  network  file  server  system  and  internal  e-mail 
system.  The  computer  elements  were  essential  in  sharing  information  among  the  teams  as  well  as  providing  an 
electronic  medium  to  store  and  retrieve  quality  and  process  information  needed  for  all  locations.  Having  up  to  date  current 
and  controlled  documents  available  to  everyone  is  an  essential  feature  and  audit  point  in  the  quality  standard.  Electronic 
media  is  very  important  in  adhering  to  the  standard  on  a global  basis  with  several  geographic  locations. 

The  development  of  the  quality  system  with  the  associated  quality  manual  is  the  first  and  most  important  step  in 
the  certification  process.  In  this  manual,  the  overall  quality  system  is  defined  and  responsibility  is  designated.  The 
organization's  system  with  respect  to  each  of  the  applicable  elements  are  defined  and  described.  Associated  documented 
processes  are  also  referenced  in  this  manual.  ISO  certification  must  be  achieved  through  an  independent  auditor.  There 
are  many  types  of  organizations  that  provide  this  type  of  service.  Once  the  certification  agency  was  selected,  they  visited 
the  locations  and  educated  the  executive  and  core  teams  on  the  certification  process.  The  certifier  then  reviews  your 
quality  manual  before  you  go  though  the  ISO  audit.  The  quality  manual  is  really  the  basis  for  your  certification  audit. 

The  development  of  the  quality  manual  proved  to  be  an  interesting  and  time  consuming  part  of  the  process. 

There  were  two  issues  to  balance  in  the  developmentof  this  manual.  The  issues  were  a central  corporate  thrust  on  the 
quality  direction  as  well  as  local  processes  at  each  site.  There  was  quite  a bit  of  dialogue  as  to  what  processes  were 
common  and  unique  to  each  site.  The  result  was  a divisional  manual  as  well  as  each  site  developing  a manual  for  its 
unique  processes.  The  ISO  certification  process  is  handled  by  each  site  with  each  location  being  certified.  One  important 
step  in  ensuring  success  is  to  have  a person  that  is  your  site  internal  auditor  so  that  the  processes  can  be  audited  before 
the  final  audit. 

A number  of  processes  were  developed  and  adapted  as  a result  of  the  ISO  process.  One  item  that  was  significant 
to  the  R&D  environment  was  requirements  analysis.  Documenting  and  communicating  what  the  actual  requirements  arc 
for  a software  and  hardware  development  project  was  an  important  addition.  As  a result,  the  development  process  included 
what  was  called  Product  Development  Quality  Plans  (PDQP's)  for  each  major  design  effort.  Processes  were  set  up  to 
review  and  sign  off  these  documents  with  all  appropriate  groups.  The  PDQP  was  a link  to  customer  requirements.  The 
documents  forced  more  communication  with  customers. 
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Training  was  essential  to  the  certification  process.  There  were  both  formal  and  informal  elements.  The 
independent  certification  audit  is  a test  of  everyone’s  understanding  of  the  process  as  well  as  whether  the  defined  processes 
are  in  place.  During  certification,  an  auditor  randomly  selects  individuals  in  the  organization  and  interviews  them.  They 
are  questioned  on  the  processes  they  are  following,  how  they  get  access  to  the  quality  records  that  contain  the 
organization's  quality  position,  as  well  as  the  elements  and  processes  that  apply  to  their  job.  Training  first  addressed  the 
ISO  process  itself.  There  was  training  at  each  site  on  its  quality  system  and  the  process  used  by  employees  to  obtain 
quality  records.  Employees  also  had  to  be  trained  in  determining  if  they  were  accessing  the  most  current  version  of 
documents.  Training  all  employees  to  access  documents  electronically  proved  beneficial. 

Training  in  the  ISO  system  is  also  necessary  to  ensure  that  all  employees  are  qualified  to  do  their  jobs.  Prior  to 
the  ISO  implementation,  training  completion  records  were  kept  for  completed  courses.  There  was,  however,  no  formal 
process  of  needs  assessment  tied  to  job  qualifications.  The  ISO  system  forced  the  division  to  examine  jobs/position 
profiles  and  determine  the  requirements  for  each  job.  As  mentioned  above,  the  ISO  standard  does  not  say  that  every 
individual  has  to  meet  the  requirements  initially,  but  a training  or  development  plan  has  to  be  in  place  for  a person  to 
meet  those  qualifications.  A global  curriculum  committee  was  established  to  determine  the  needed  skills  in  each  function 
in  each  lab.  The  skills  included  both  technical  and  personal  development  skills.  Training  profiles  were  developed  for 
individuals.  The  training  departments  at  each  location  worked  with  line  managers  to  ensure  that  each  employee 
completed  a training  profile  with  their  manager.  The  profiles  were  signed  on  a hard  copy  by  the  manager  and  employee. 
The  profiles  were  summarized  by  the  training  function  to  develop  the  overall  training  plan  for  each  lab.  The  training 
plans  together  with  other  needs  assessment  tools  helped  in  determining  training  budgets  and  appropriate  types  of 
instruction.  The  initial  job  descriptions,  usually  kept  by  Human  Resources,  are  important  for  audit  purposes.  The 
auditor  will  examine  the  job  descriptions  first  and  then  examine  random  employee  profiles  to  determine  if  the 
qualifications  and  development  plans  match. 

After  approximately  a year  of  effort,  all  R&D  lab  sites  achieved  certification  on  the  first  audit.  The  sites  were 
also  recertified  o^e  year  later.  After  the  first  year,  the  division  went  though  a massive  reorganization  linking  marketing, 
product  line  management,  and  systems  engineering  into  a common  business  unit  to  improve  customer  effectiveness. 

The  certification  process  now  expanded  to  include  the  entire  business  unit.  The  executives  in  this  business  unit  viewed 
the  ISO  process  as  one  way  to  establish  the  organizational  links  and  teaming  to  achieve  this  business  integration.  The 
ISO  process  proved  an  effective  way  to  do  this.  A successful  audit  was  completed  for  the  entire  business  unit  less  than 
six  months  ago.  There  were  several  initiatives  established  to  link  the  R&D  effort  directly  to  customers.  In  a recent  Core 
Quality  Team  meeting,  the  ISO  process  was  recognized  for  its  contribution  to  the  reorganization. 

Comments  from  the  independent  audit  group  were  very  positive  and  highlighted  important  elements  of  a quality 

system: 


You  have  a unique,  integrated  quality  system,  which  is  among  the  best  I have  ever  seen.  It  is  well  structured, 
well  thought-out  and  well  applied  to  meet  the  ISO  9001  requirements. 

This  is  the  first  time  I have  see.i  a Design  group  take  a lead  in  customer  interaction  by  talking  to  customers  at 
the  early  stages  of  the  design. 

Your  Quality  System  Reviews  (QSRs)  are  truly  impressive  in  the  fact  that  it  covers  the  entire  scope  of  the 
Business  Indicators. 

You  have  effectively  applied  the  Quality  System  to  clarify  Roles  and  Responsibilities  between  ...  functional 
groups. 

The  important  themes  were  linking  of  functional  groups  and  the  link  to  the  customer.  Training  can  have  a role  in  this 
effort  by  designing  learning  experiences  that  link  the  business  group  to  the  customer.  In  this  particular  division, 
seminars  and  field  trips  are  made  to  customer  sites  as  often  as  possible  to  help  firm  up  the  link  to  the  customer. 
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Role  of  Information  Technology 

The  ISO  standard  really  is  a method  of  information  exchange.  It  is  a method  for  documenting  processes  to 
empower  everyone  in  an  organization  to  meet  customer  expectations.  The  role  of  information  technology  proved 
essential  to  this  communication.  Documents  were  continually  developed,  reviewed,  and  electronically  exchanged,  in  an 
information  network.  The  electronic  file  storage  provided  the  required  vintage  control  so  that  everyone  worked  from  the 
same  version.  One  factor  that  must  be  considered  in  utilizing  this  type  of  information  network  is  the  selection  of  the 
software  and  hardware  platforms.  In  this  case,  framemaker  was  used  for  all  ISO  documents  because  it  could  be  sent 
through  a variety  of  • omputer  hardware  platforms  such  as  Macintosh,  DOS,  and  UNIX  without  many  conversions  and 
reformatting.  Another  lesson  learned  through  this  process  was  designing  for  useability  by  all  employees  on  the  network 
servers  as  well  as  providing  training  on  accessing  information. 

Reducing  Costs 

Implementing  the  ISO  process  during  major  budget  cuts  in  an  organization  is  a challenge,  particularly  when  it  is 
done  globally.  As  mentioned  previously,  utilizing  video  meetings  and  conference  calls  will  reduce  the  needs  for  travel. 

An  important  method  that  will  help  to  keep  costs  to  a minimum  is  to  develop  and  keep  processes  as  simple  as  possible 
while  still  meeting  the  intent  of  the  standard.  Realize  that  processes  will  change  and  will  be  revised.  In  terms  of 
training,  an  extensive  curriculum  based  system  is  costly.  Training  is  becoming  more  performance  based.  Performance 
analysis  should  be  done  with  employees.  There  may  also  be  a need  for  basic  competencies  training  in  addition  to 
performance  issues. 

Hale  and  Westgaard  (1995)  outline  some  simple  steps  in  process  development  that  impact  the  budget: 

• Agree  what  the  steps  are. 

• Reach  consensus  on  what  steps  are  necessary. 

• Specify  what  resources  (e.g.,  time,  equipment,  information)  each  step  of  the 
process  and  the  process  as  a whole  require. 

• Evaluate  which  resources  add  value,  are  used  well,  are  redundant,  or  reduce 
quality. 

• Determine  cycle  time. 

• Agree  on  what  information  is  required  and  what  is  nice  to  know. 

• Come  to  consensus  about  how  information  is  used,  where  it  comes  from,  and 
what  information  is  missing,  late,  or  complete. 

The  cost  for  ISO  certification  is  mainly  internal  except  for  the  expense  of  the  independent  auditor.  The  typical  cost  for 
the  independent  audit  group,  for  the  initial  certification,  is  approximately  25K  and  approximately  10K  per  year  after  that. 
The  above  case  was  for  a location  of  approximately  300  people.  The  external  auditor  goes  through  all  the  quality 
records,  thus  how  well  these  records  are  organized  will  impact  cost.  The  internal  cost  consisted  of  approximately  a dozen 
people  spending  10-20  percent  of  their  time  on  the  ISO  process.  Training  and  understanding  in  the  ISO  standards  will 
minimize  this  expense. 

Conclusion 

The  benefits  of  the  ISO  certification  become  more  apparent  after  certification.  Employees  begin  to  realize  later 
that  the  ISO  process  will  actually  simplify  their  work  (Todorov,  1996).  Achieving  the  standard  in  organizations  is  an 
indication  that  information  flow  and  training  are  greatly  improved.  Ultimately,  there  is  a decrease  in  functional  defects  in 
a company,  a reduction  of  nonconformity's,  and  greater  discipline  in  process  execution. 

There  have  been  instances  where  productivity  improvements  have  been  documented  due  to  the  ISO  standard.  The 
management  model  proposed  by  the  ISO  standard  may  in  some  cases  permit  a decrease  in  the  manager/employee  ratio  as  a 
result  of  autonomous  work  teams  and  greater  responsibility  given  to  employees.  By  empowering  people,  senior 
management  is  relieved  of  the  management  of  daily  activities  and  can  concentrate  on  more  strategic  aspects  of  the 
business.  The  ISO  standard  provides  this  improved  empowerment  model  by  ensuring  information  on  the  companies 
quality  system,  customer  links,  and  processes  are  documented  and  are  available  to  everyone.  Employees  have  at  their 
disposal  concrete  tools  (training,  work  instructions,  forms,  etc.)  to  perform  quality  work  (Todorov,  1996).  Instructional 
and  information  technology  can  facilitate  the  implementation  of  the  ISO  standard  on  a global  basis. 
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The  Common  Thread  Case  Project  is  a multi-year  project  that  uses  multimedia  compact  disc  technology  and 
high-bandwidth  telecommunications  to  provide  a unique,  case-based  professional  development  network  for  teachers  in  the 
midst  of  systemic  reform  in  the  state  of  Kentucky. 

The  project,  now  in  its  second  year,  is  a partnership  between  the  University  of  Kentucky,  the  University  of 
Louisville,  and  Kentucky  Educational  Television.  Year  one  of  the  project  centered  on  the  development  of  a series  of  five 
secondary  and  five  elementary  cases,  available  in  both  hard  copy  and  CD-ROM.  Each  case  in  the  series,  called  The 
Common  Thread  Cases,  is  a true  story  and  captures  the  dilemmas  and  accomplishments  of  teachers  involved  in  reform. 
The  criteria  for  the  Common  Thread  Cases  (CTC)  have  been  fully  described  elsewhere  (Bliss  & Mazur,  1995a).  However, 
the  case  criteria  are  salient  to  the  present  discussion  and  are  reiterated  in  Appendix  A.The  cases  are  designed  for  in-depth 
discussions  that  critically  analyze  pedagogy  and  promote  reflection.  Year  two  of  the  project  focuses  on  training  and 
dissemination  of  the  case  materials.  The  intent  of  the  training  is  to  provide  teachers  with  case  facilitation  skills  and 
technology  to  develop  and  support  a professional  community  of  teachers.  In  addition  to  case  facilitation  training,  the 
project  also  is  working  with  participants  to  explore  telecommunications  for  ongoing  professional  development  and 
collaboration.  Currently,  we  are  examining  two  questions  potentially  crucial  to  the  project's  long-term  success. 

(1 ) To  what  extent  can  the  cases,  along  with  accompanying  study  tools  and  resources  and  available 
telecommunications  technology,  become  the  nexus  of  a larger  system  of  knowledge  support  that  is  essential  for  teacher 
professionalism? 

(2)  What  are  the  dimensions  of  such  a knowledge  support  system? 

Background  and  Need  for  Knowledge  Support 

Kentucky  teachers  face  unique  challenges.  Now  in  its  fifth  year  of  implementation,  the  Kentucky  Education 
Reform  Act  of  1990  (KERA)  requires  comprehensive  reform  in  governance,  finance,  and  curriculum.  The  reform  includes 
an  extensive  technology  initiative,  the  Kentucky  Educational  Technology  System.  The  purpose  of  the  statewide  network 
is  to  improve  instruction  and  support  teacher  professional  development  (Kentucky  Master  Plan  for  Technology,  1992) 
through  the  use  of  telecommunications  and  on-line  resources.  Schools  are  now  required  to  provide  high-end  desktop 
computing  in  classrooms.  An  integrated  wide-area  network  will  connect  all  the  states'  school  districts,  colleges  and 
universities,  and  governmental  agencies.  Such  an  extensive  network  in  a largely  rural  state  is  intended,  in  part,  to  support 
the  profe  rional  needs  of  teachers  - teachers  who  are  expected  to  engage  in  new  types  of  teaching  (e.g.,  portfolio 
assessment)  and  problem-solving  activities.  For  novice  teachers,  performance  is  now  assessed  through  portfolios  in 
which  they  demonstrate  what  they  know  and  are  able  to  do  consistent  with  state  adopted  performance  standards. 

Several  recent  studies  of  the  KERA  implementation  have  stressed  the  need  for  more  strategic  professional 
development  approaches  (Robertson  Associates,  1995).  Teachers  want  more  examples  of  new  practices  and  ongoing 
support  for  incorporating  new  approaches  (Mathews,  1995;  Daniels  & Stallion,  1995).  Moreover,  to  be  more  effective  in 
meeting  the  needs  of  teachers,  technology  should  be  integrated  into  flexible  professional  development 
(Mazur,  1995). 


Cases  and  Teacher  Professional  Development 

Cases  have  been  used  for  teacher  professional  development  for  some  time  and  for  a variety  of  purposes 
(Wasserman,  1993;  Kagan,  1993;  Kleinfeld,  1989).  Among  the  most  prominent  uses  of  the  case  approach  are  (1)  to 
support  collaborative  work  between  experts  and  novices  and  to  compare  differences  in  novice  and  expert  approaches 
(Borko  et  al.,  1992;  Borko  & Livingston,  1989),  (2)  to  frame  situational  learning  (Koschman  et  al,  1990)  and  (3)  to 
examine  expert  decision  making  in  complex,  ill-structured  domains  (Spiro  et  al.  1988).  In  addition,  cases  contextualize 
problem  solving?  (Cognition  and  Technology  Group,  1992,  1991;  Risko,  1992b;  Risko,  Vount  & Towell,  1991). 
However,  electronic  cases  for  educators  (Fishman  & Duffy,  1992,  1993;  Desberg,  Colbert  & Trimble,  1995)  have  been 
developed  only  recently. 

Each  Common  Thread  Case  CD-ROM  contains  the  entire  discursive  text  of  the  hard  copy  case  transformed  into 
a multimedia  "animated  narrative.  That  is,  the  text  is  augmented  with  accompanying  non-discursive  descriptive  material 
consisting  of  high  quality  graphics,  video  clips  and  audio  narration.  These  enhancements  are  intended  to  provide  teachers 
with  multiple  ways  of  knowing  the  subject  (Eisner,  1993).  Strategic  case  content  locators  and  graphical  overviews 
provide  several  ways  to  navigate  through  the  case  content.  A variety  of  tools  and  resources  designed  to  supplement  study 
and  discussion  of  the  case  are  also  provided:  Case  discussion  questions,  exhibits  such  as  lesson  plans  and  examples  of 
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students'  work,  case  commentaries  by  teachers,  administrators,  and  scholars,  a case  bibliography,  and  relevant  articles  are 
included.  Each  CD  contains  telecommunications  shareware  that  facilitates  the  use  of  on-line  networks  to  discuss 
questions  or  issues  related  to  the  case.  A World  Wide  Web  homepage  for  the  project  is  also  available 
(http://www.uky.edu/~borchyi/KERA.html). 

Knowledge  Support  and  Performance  Outcomes  for  Teacher  Professionalism 

The  Common  Thread  Cases  are  consistent  with  a proposed  approach  to  upgrade  teachers'  skills  is  through 
integrating  powerful  new  technologies  with  teacher  professional  development  (Barron  and  Goldman,  1994)).  However, 
many  of  the  problems  related  to  technology  and  productivity  identified  by  business  and  industry  have  resurfaced  in 
professional  development  for  teachers  (Lieberman  & McLaughlin,  1992).  For  example,  educators,  like  their  counterparts 
involved  in  training  for  business  and  industry,  have  seen  the  poor  results  of  one-shot,  one-size-fits-all  training  that  does 
not  provide  opportunities  for  ongoing  practice  and  feedback  on  performance  (Savage,  1994).  To  combat  this  problem  new 
systems,  termed  on-line  performance  support  systems,  have  been  designed  to  link  innovative  applications  to  new 
learning,  working,  and  collaboration  paradigms  (Schoenmaker,  1993).  These  systems  have  been  described  as  continuous 
learning  environments  (Forman  & Kaplan,  1994).  That  is,  the  system  provides  an  array  of  integrated  information 
resources  such  as  references,  guidance,  or  tools  on  the  desktop  that  are  under  the  users'  control.  Delivered  via  computer, 
this  new  generation  of  electronic  performance  support  systems  (Stevens  and  Stevens,  1995),  provides  immediately 
needed,  task-  and  situation-  specific  information  that  is  accessible  from  electronic?  networks  and/or  multimedia  databases. 
Using  tailored  navigational  and  indexing  utilities,  these  systems  can  store  vast  amounts  of  information  and  embed 
training  (Mullins,  1989)  that  can  meet  the  needs  of  many  users  who  may  need  the  information  for  different  reasons.  The 
untapped  strength  of  these  new  interactive  and  networked  technologies  lies  in  their  potential  to  deal  with  complexity  and 
sort  out  multiple  factors  involved  in  a problem  (Savage,  1994). 

What  is  the  potential  of  these  continuous  learning  environments  for  professional  development  in  education? 
Despite  some  congruence  with  approaches  generally  termed  constructivist  (Duffy  & Jonassen,  1993,  Wilson,  1995),  there 
remains  significant  conceptual  differences  between  programs  designed  to  support  educational  endeavors  and  the  continuous 
learning  environments  used  to  support  productivity  in  business  and  industry.  One  is  the  nature  of  the  cognitive  tasks 
involved  in  productive  work.  Teaching  requires  professional  judgments  regarding  actual  practice  or  technique  that  may 
vary  widely  in  specific  circumstances.  Teachers'  judgments  requires  what  has  been  termed  narrative,  rather  than 
paradigmatic,  knowing  (Bruner,  1991).  There  is  no  one  set  of  prescribed  rules  or  procedures  that  will  work  in  any  given 
set  of  circumstances. 

Another  aspect  of  the  educational  situation  that  is  significantly  different  from  business  environments  is  that 
both  novices  and  veterans  are  assigned  similar  tasks  regardless  of  experience.  That  is,  a first  year  teacher  and  a twenty-year 
veteran  may  both  be  teaching  a class  of  mixed  ability  second  graders  in  an  urban  school.  While  each  has  much  to  learn 
from  the  other,  the  time  and  opportunity  are  usually  lacking  to  capitalize  on  the  possibilities  for  mentoring  or  peer 
assistance  (Little,  1991;  Raney  and  Robbins,  1989).  In  addition,  the  traditional  isolation  of  teachers  has  inhibited  the 
sharing  of  pedagogical  insights  gained  from  experience. 

The  strategic  role  of  dialogic  conversation  in  enabling  participants  to  create  commonly  shared  meaning  and  to 
"transform  their  mindsets  and  think  about  change  and  education"  has  been  recently  conceptualized  by  Jenlink  and  Carr 
(1996  p.  34).  Furthermore,  Brown  & Campione  (1990)  have  described  how  essential  a community  of  learners  is  to 
supporting  critical  thinking,  reflection,  and  change.  Specifically,  networks  of  engaged  participants  can  become  the  much- 
needed  contexts  for  transforming  practice. 

Currently,  there  are  a plethora  of  on-line  services  for  teacher  networking  such  as  on-line  discussion  groups, 
bulletin  boards,  and  extensive  databases  for  information  retrieval.  But  many  users  complain  that  electronic  information  is 
overwhelming,  unfocused,  or  difficult  to  access  (Harasim,  1990)  and  this  perception  has  seriously  limited  the  use  of  on- 
line resources  for  professional  development.  In  their  efforts  to  change  practice,  teachers  need  to  engage  in  intensive,  more 
structured  professional  communities  and  intellectual  teamwork  (Gallegher  & Kraut  1991)  than  is  now  available.  The 
Common  Thread  Case  project  seeks  to  begin  addressing  this  issue  through  the  integration  of  multimedia  cases  and 
telecommunications  in  an  open,  somewhat  unstructured  knowledge  support  system. 


Dimensions  of  An  Open  Knowledge  Support  System 


The  knowledge  support  environment  has  to  be  robust  enough  to  encourage  thought  and  promote  insight,  open 
enough  to  allow  users  to  raise  topics  that  are  personally  relevant,  and  yet  sufficiently  focused  to  emphasize  state-adopted 
teacher  performance  standards.  The  design  of  the  knowledge  support  system  must  include  three  dimensions.  These  are: 

(1)  An  Instrumental  Dimension.  The  core  content  (the  cases)and  information  contained  in  the  resources  must  be  rich 
enough  to  accommodate  novice  and  experienced  educators  and  appeal  to  a wide  variety  of  experiences.  Simultaneously, 
the  performance  standards,  consistent  with  national  standards,  must  play  a prominent  role.  The  design  and  presentation  of 
information  provides  opportunities  for  studying  complex  pedagogy  and  offers  multiple  ways  to  access  that  information. 
Furthermore,  the  system's  design  should  incorporate  aesthetic  aspects  (Saito  et  al.  1995)  that  capitalize  on  the  expressive 
potential  of  multimedia  technology  to  motivate  and  engage  the  educational  imagination  described  by  Eisner  (1986).  The 
system  also  needs  to  be  flexible  and  accessible  to  accommodate  teachers*  scheduling  demands. 

(2)  A Relational  Dimension.  To  enable  the  teacher  to  thoroughly  analyze  various  aspects  of  a case  as  it  relates  to 
teaching  standards  as  well  as  to  theory  and  practice,  the  knowledge  support  system  makes  available  integrative  tools  and 
resources.  Examining,  gathering,  and  linking  information  may  involve  questioning  or  clarifying  the  case  situation, 
identifying  key  points,  or  seeking  further  information  about  case  topics.  Tools  to  help  teachers  elaborate  on  issues  raised 
and  double  checking  information-all  of  which  are  instantly  available  within  the  case  program,  comprise  a relational 
dimension  to  knowledge  support  system  that  is  structured  not  by  the  program's  sequence  but  by  the  questions,  interests, 
or  concerns  of  the  user. 

(3)  A Communication  Dimension.  The  system  incoporates  tools  that  encourage  and  enable  teachers  to  communicate  their 
analysis  of  the  case  issues  to  others  who  have  also  used  the  case.  Teachers  can  revise  and  create  new  ideas,  capitalize  on 
the  experiences  of  others  who  may  have  tested  ideas  from  the  case  and  engage  in  the  critical  discourse  so  essential  to  the 
development  of  professional  communities. 

The  chart  below  illustrates  how  prominent  case  features  and  telecommunications  tools  shown  in  Appendix  B 
comprise  various  dimensions  of  the  open  knowledge  support  system. 
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The  program  has  been  field  tested  with  approximately  fifty  new  and  experienced  Kentucky  teachers.  The  most 
obvious  issues  emanating  from  the  field  tests  concern  the  communications  dimension  of  the  knowledge  support  system. 
Not  surprisingly,  users  were  uncomfortable  with  initiating  on-line  video  conferences  with  strangers.  Also,  the  notion  of 
on-going  professional  development  carried  out  via  electronic  mail  was,  in  fact,  so  foreign  to  most  teachers  that  they  had 
difficulty  incorporating  contacts  with  other  teachers  into  their  routines.  Just  as  workers  in  business  and  industry  had  to 
adjust  work  habits  to  truly  utilize  the  highly  accessible  resources  in  continuous  learning  environments,  teachers 
apparently  need  encouragement  and  follow-up  on  the  use  of  on-line  forums  and  databases  to  support  their  work  and 
professional  development  activities.  Most  importantly,  the  issue  of  sustainability  was  raised.  What  is  necessary  to 
maintain  professional  conversation  and  community  once  it  has  been  initiated? 
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At  this  point  in  the  implementation  of  the  Common  Thread  Case  project,  the  communications  dimension 
needs  careful  investigation.  As  our  research  proceeds  we  will  be  examining  which  resources  are  most  suited  to 
supporting  aspects  of  professional  discourse.  For  example,  will  features  of  teleconferencing  be  more  valuable  for  certain 
types  of  discussions,  or  only  as  the  initial  contacts  for  collaboration?  "What  kinds  of  discussion  will  be  best  suited  for  e- 
mail?  Will  the  World  Wide  Web  forum  or  two-way  video  conferencing  be  the  most  effective  model  for  study  group 
discussion  ? What  additional  resources,  not  yet  available  as  part  of  the  existing  system,  will  teachers  need  and  want?  As 
refinements  to  the  development  of  the  communications  dimension  of  the  knowledge  support  system  proceed,  we  hope  to 
develop  dynamic  models  for  open,  somewhat  unstructured  communities  that  empower  teachers  in  their  efforts  to  improve 
practice  through  conversation  and  community. 
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Appendix  A 


The  Common  Thread  Case  Criteria 

Each  Common  Thread  Case  contains  four  elements: 

1 . A well  formulated  narrative  structure  (Connelly  & Clandinin,  1 990) 

2.  A true,  factually  correct,  compelling  acocunt.  The  authenticity  of  the  case  is  essential  and  strategic,  as 
suggested  by  Phillips  (1994),  who  notes  that  one  is  more  likely  to  accept  what  is  true  and  more  likely  to  be  successful 
when  what  one  acts  upon  is  correct. 

3.  Tangible  episodes  of  good  teaching.  Teaching  episodes  must  include  the  particulars  that  become  tangible  to 
the  reader.  The  importance  of  rich  contextual  details  has  been  thoughtfully  described  elsewhere  (Eisner,  1 990). 

4.  Consequential  Aspects  of  Standards-Based  Practice.  Each  case  includes  events  that  are  relevant  to  teacher 
performance  standards.  These  standards  are  not  models,  but  frames  of  reference.  The  holistic  nature  of  teaching  implies 
that  various  standards  will  often  apply  and  even  overlap  in  the  same  case. 
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Introduction 


Many  instructional  designers  are  concerned  about  the  difficulties  with  disseminating  Instructional  Systems 
Design  (ISD)  models  to  teachers  who  are  believed  to  be  potential  users  of  ISD  (Burkman,  1987).  Often  lamenting  that 
they  do  not  seem  to  impact  the  public  schools,  instructional  designers  believe  that  exposing  pre-service  and  in-service 
teachers  to  ISD  procedures  and  products  is  a very  important  and  first  step  in  turning  the  situation  around  (Driscoll,  1989). 

However,  educational  literature  reveals  few  attempts  to  relate  instructional  design  theory  and  methods  to  teaching 
practice.  While  research  in  teacher  thinking  presents  a new  image  of  teachers  and  teaching  practice,  very  few  of  these 
findings  are  reflected  in  instructional  design  theories  and  models. 

The  purpose  of  this  paper  is  to  propose  a new  conceptual  model  for  thinking  about  teaching  that  incorporates 
current  findings  of  research  on  teachers’  thinking  and  components  of  instructional  design  models  and  principles.  The 
paper,  therefore,  will  fo  us  on  reviewing  the  major  findings  of  research  on  teacher  thinking  and  instructional  systems 
design.  A conceptual  model  will  then  be  presented  to  bring  the  two  closely  related  fields  together.  The  potential 
implications  of  the  model  for  instructional  development  and  research  in  instructional  design  and  teacher  thinking  will  also 
be  discussed. 

Instructional  Systems  Design  Models  and  Principles 

Instructional  systems  design  draws  its  models  and  principles  from  a variety  of  disciplines  mainly:  general 
systems  theory,  communications  theory,  learning  theory,  and  instructional  theory  (Richy,  1986).  Instructional  systems  is 
defined  as  an  arrangement  of  resources  and  procedures  used  to  promote  learning  (Gagne,  Briggs,  & Wager,  1988).  This 
definition  uses  the  term  instructional  planning  as  a complete  process  of  analysis,  design,  development,  implementation 
and  evaluation.  A number  of  models  have  been  developed  by  designers  during  years  of  instructional  design  practices  (see 
Andrew  & Goodson,  1980).  Design  models  come  from  industry,  education,  the  military  branches,  and  a variety  of  other 
sources.  These  models  define  what  should  be  and  prescribe  the  necessary  activities  and  sometimes  prescriptions  for 
effective  instruction.  Some  of  these  models  are  called  "conventional  models”  and  focus  on  the  instructional  systems  level 
with  a closed  system  approach  to  design  (Banathy,  1987).  Complementary  to  the  conventional  (micro)  instructional 
design  models  are  macro-design  or  systems  models  that  have  a different  purpose,  different  systemic  characteristics,  and  use 
a different  design  approach.  The  purpose  of  these  models  is  to  provide  societal-based  systems  and  arrangements  of 
learning  resources  and  connect  these  systems  and  arrangements  with  the  learning  experience  level  of  education. 

Despite  variability  among  different  conventional  instructional  design  models,  they  are  all  characterized  by  three 
key  features:  (1)  a linear  planning  process,  (2)  an  objective-first  approach  to  planning,  and  (3)  a generic  model  for 
planning  instruction.  These  models  usually  begin  with  specifying  instructional  goals,  developing  a list  of  specific- 
objectives,  developing  test  items  for  each  objective,  and  developing  instructional  strategies.  After  the  implementation  of 
instruction,  the  next  step  is  to  evaluate  the  plan  to  examine  its  effectiveness.  The  last  and  most  important  step  is  to 
review  the  instruction  based  on  the  performance  information  and  to  revise  it  as  required. 

The  effectiveness  of  the  different  aspects  of  conventional  instructional  design  models  are  supported  by  teacher 
effectiveness  research  and  research  on  cognitive  processes  of  learning.  The  effectiveness  of  an  educational  program  that 
is  designed  by  the  complete  instructional  design  models  (macro  and  micro)  has  also  been  investigated  and  supported. 

Instructional  Design  Models  and  Images  of  Teachers  and  Teaching 

Since  the  establishment  of  the  instructional  systems  design  field,  a major  effort  of  experts  in  the  field  has  been 
to  use  ISD  models  and  principles  to  improve  education  at  public  school.  Generally  speaking,  two  approaches  have  been 
primarily  used  to  disseminate  ISD  models  in  public  schools.  The  first  approach  has  been  to  provide  school  systems  with 
systematically  designed  curriculum  and  materials  to  be  implemented  by  teachers.  The  teacher-training  programs  and/or 
teachers’  guides  would  be  centered  on  the  development  of  necessary  skills  needed  by  teachers  to  implement  the 
systematically  designed  curriculum.  This  approach  was  primarily  based  on  the  designers'  image  of  teachers  as  "delivery 
systems"  (Tosti  & Ball,  1969),  which  later  changed  to  the  image  of  "teachers  as  managers"  (Taguchi,  1993). 
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The  second  approach  centered  on  the  idea  of  teachers  as  designers  and  developers  of  their  own  instruction 
(Taguchi,  1993).  This  approach  aimed  to  empower  teachers  to  be  actively  involved  in  their  curriculum  design.  On  the 
basis  of  this  approach,  teachers  would  design  their  instruction  using  fundamentals  of  instructional  design  models  to  plan, 
evaluate  and  modify  instruction  as  a regular  and  on-going  process  and  part  of  their  classroom  instruction  (Dick  & Resier, 
1989). 


However,  none  of  the  above  approaches  are  found  to  be  successful  in  disseminating  ISD  models  and  theories  at 
public  schools.  Research  on  curriculum  implementation  showed  that  in  spite  of  the  designers1  efforts  at  in-service  and 
pre-service  training,  teachers  failed  to  implement  instructional ly  designed  products  (Back  & McCombs,  1984,  cited  in 
Burkman,  1987).  Although  the  researchers  linked  the  lack  of  success  to  the  poor  dissemination  efforts  and  lack  of 
support  for  teachers  who  used  the  materials,  critics  questioned  the  assumptions  that  the  teacher  is  a passive  recipient  of 
educational  products,  and  that  a technology  of  teaching  is  transferable  from  one  situation  to  another.  Research  in  staff 
development  and  teacher  decision  making  also  supported  these  suspicions  (Posner,  1995). 

While  instructional  designers  had  some  success  in  providing  training  and  practice  in  instructional  design  and 
theories  in  teacher  education  programs  (Snelbecker,  1987),  the  available  information  on  teachers  and  their  use  of 
instructional  design  practice  is  not  encouraging  either.  Some  of  the  research  on  teacher  planning  and  decision  making 
processes  (e.g.,  Shavelson,  1983;  Brown,  1988)  reveal  that  teachers  typically  do  not  plan  and  provide  instruction  in 
accordance  with  procedures  taught  in  teacher  preparation  programs.  Yet,  other  studies  have  shown  that  experienced 
teachers  do  believe  that  rationale  models  are  to  be  taught  to  novices  (e.g.,  Neale,  Pace  & Case,  1983)  and  they  do  attend 
to  learning  outcomes,  sometimes  during  teaching  and  sometimes  after  interactive  teaching  is  over  (McLead,  1981). 

This  discrepancy  between  teachers1  non-use  of  systematic  models  and  their  favorable  attitude  toward  them  raises 
many  issues  regarding  ISD  models.  A number  of  reasonable  explanations  have  been  identified.  Educational 
technologists  (e.g.,  Tessmer  & Wedman,  1990)  have  begun  to  acknowledge  the  complexity  of  design  and  development 
processes,  and  they  have  described  ways  of  adjusting  this  complexity  to  the  contexts  in  which  such  processes  are 
employed.  However,  despite  the  tremendous  explosion  of  knowledge  derived  from  research  on  teachers1  thinking  and 
teaching  over  the  last  two  decades,  very  little  of  this  emerging  knowledge  concerns  ISD.  As  the  gap  between  teacher- 
thinking research  and  instructional  design  models  and  principles  becomes  wider,  the  possibility  that  ISD  models  will  be 
used  in  school  systems  becomes  lessened. 

In  an  attempt  to  relate  findings  of  research  on  teacher  thinking  to  instructional  design  models  and  practice,  in  the 
following  section  I present  a synthesis  of  research  on  teacher  thinking.  I then  use  the  findings  of  this  research  to  propose 
a conceptual  framework  for  thinking  about  teaching  considering  instructional  design  models  and  principles. 

From  Teachers  as  Decision  Makers  to  Teachers  as  Sense  Makers:  A Synthesis  of  Research  on  Teacher  Thinking 

Much  of  the  early  research  on  teachers1  thinking  and  decision  making  was  based  on  an  analogy  between  teachers1 
diagnoses  and  medical  diagnoses  (e.g.,  Barrows  & Bennett,  1972;  Elstein,  Shulman,  & Sprafka,  1978).  It  was  believed 
that  the  major  role  of  teachers  was  to  diagnose  children’s  difficulties  and  progress  and,  on  the  basis  of  the  diagnoses, 
prescribe  effective  and  appropriate  learning  tasks  for  them.  This  image  of  the  teacher  was  influenced  by  theories  in 
cognitive  psychology,  which  in  turn  was  influenced  by  a communication  information-processing  model.  This  influence 
led  to  research  on  teachers1  thinking  in  1975  that  assumed  parallel  cognitive  processes  between  teachers  and  physicians 
(e.g.,  Fogarty,  Wang  & Creek,  1982;  Marland,  1977;  Morine  & Vallance,  1975).  The  main  focus  of  many  of  these 
studies  was  on  the  structure  and  content  of  teachers1  thoughts  and  sometimes  their  cognitive  processes. 

However,  many  researchers  have  sought  to  demonstrate  a close  parallel  between  teachers'  thought  processes  and 
specific  models  of  thinking,  especially  the  decision-making  model  (Clark  & Peterson,  1986;  Shavelson  & Stern,  1981). 
Early  models  of  the  decision-making  process  among  teachers  implied  a linear  course  of  action  with  alternative  branches, 
which  was  very  similar  to  the  models  of  diagnostic  problem  solving  in  medicine  (see  Kagan,  1988).  In  general,  this 
body  of  research  has  focused  on  cognitive  processes,  rather  than  on  the  more  general  knowledge  which  guides  the  practice 
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of  teaching  in  complex  classroom  situations  (Borko,  Livingston,  McCaleb,  & Mauro,  1988;  Clark  & Peterson,  1986; 
Marland  & Osborne,  1990;  Wood,  1988). 

In  the  past  fifteen  years,  researchers  in  the  field  of  teachers’  cognition  have  grown  increasingly  sensitive  to  the 
importance  of  classroom  ecology  in  attempting  to  identify  teachers’  problem-solving  strategies  (Kagan,  1988).  Once 
researchers  began  to  look  closely  at  the  ecology  of  the  classroom,  it  became  clear  that  teachers  work  in  a context  that  is 
complex  in  terms  of  multiple  activities  and  continual,  unpredictable  change.  Mechanistic  decision-maker  frameworks 
seemed  inappropriate  to  explain  how  solutions  were  found  for  the  myriad  of  complex  practical  problems  continually 
confronting  teachers  within  their  classrooms  and  school  communities  (Carter,  1990;  Johnston,  1993).  The  volume  of 
differential  knowledge  and  the  rapid  pace  with  which  teachers  must  access  this  knowledge  suggested  a highly  specialized 
form  of  clinical  problem  solving.  Along  with  this  shift,  teacher  cognition  has  developed  into  a wider  concept  of  "teacher 
thinking". 

Hence,  in  recent  years  the  image  of  the  teacher  as  decision  maker  has  been  replaced  with  an  image  in  which 
"sense-making”  is  the  central  cognitive  activity  of  teachers  (Kagan,  1988;  Clark  & Peterson,  1986;  NCRTE,  1988; 
Freeman,  1990,  1991).  In  this  view  teachers  not  only  make  decisions,  but  they  engage  in  several  activities,  including 
decision  making,  in  order  to  make  meaning  for  themselves  and  their  students  (Clark,  1986).  Thus,  the  metaphor  of 
teacher  as  physician  is  giving  way  to  the  image  of  teacher  as  sense  maker  or  reflective  professional  (Schon,  1983). 

According  to  the  metaphor  of  teacher  as  reflective  practitioner,  the  problems  of  practice  are  messy,  uncertain, 
complex  and  context-bound;  and,  therefore,  teachers  must  resolve  such  problems  by  mentally  experimenting  and 
manipulating  contextual  factors,  generating  alternative  hypotheses  about  the  problem  and  mentally  testing  them  in  order 
to  come  up  with  a discovery  that  leads  to  action  (Schon,  1987).  This  process  of  reframing  (seeing  the  situation  in  a new 
way  as  a result  of  unexpected  messages  from  practice)  (Mumby,  1986),  experimenting  (generating  hypotheses  about  the 
problem),  acting  (testing  a new  approach  to  practice)  and  reappraising  is  called  reflection-in-action  (Mumby  & Russell, 
1989).  This  image  looks  vastly  different  from  the  image  of  teaching  as  consisting  of  well-formed  instrumental  problems 
that  can  be  solved  by  applying  theory  and  techniques  derived  from  previous  research.  Thus,  professional  practice  is  not 
"theory",  but  the  process  of  developing  and  using  practical  theories  of  action. 

Tin  Idea  of  Professional  Knowledge 

It  has  been  since  1975  that  research  on  teacher  thinking  has  emerged  within  the  field  of  cognitive  psychology. 
Two  perspectives  have  existed  on  the  concept  of  teacher  knowledge:  cognitive  and  epistemological.  The  advocates  of  the 
cognitive  perspective  believe  that  teachers  are  professionals  who  make  reasonable  judgments  and  decisions  in  a complex, 
uncertain  environment  (Borko  & Shavelson,  1990).  On  this  view,  there  is  a relationship  between  thought  and  action. 
There  is  first  thought,  and  then  action.  Two  lines  of  research  in  the  past  decades  have  done  much  to  enrich  our 
understanding  of  teacher  knowledge:  expert-novice  research  and  research  on  the  content  and  domains  of  teacher  knowledge. 

Research  comparing  expert  and  novice  teachers  (e.g.,  Berliner,  1987;  Greeno,  Glaser  & Newell,  1983;  Larkin, 
McDemott,  Simon,  & Simon  1980;  Leinhardt,  1983;  Leinhardt,  Weedman  & Hammond,  1984)  demonstrates  that  the 
amount  of  knowledge  and  the  ways  in  which  experts  organize  their  knowledge  is  different  from  that  of  novices.  Research 
on  the  content  and  domains  of  teachers’  knowledge  suggest  different  categories  of  knowledge.  Shulman  (1986a) 
emphasizes  three  (3)  types  of  content  knowledge:  subject  matter  knowledge,  pedagogical  knowledge  and  curricular 
knowledge.  Clark  and  Lampert  (1986)  mention  three  (3)  categories  of  knowledge:  contextual  knowledge,  subject  matter 
knowledge,  and  knowledge  of  methods  and  inquiry.  Research  on  teacher  planning  and  interactive  thinking  is  also  seen  as 
a means  for  organizing  and  transforming  subject  matter  knowledge  and  curriculum  into  pedagogically  useful  forms  of 
routine  (Clark.  1986).  Both  the  research  on  the  expert-novice  teacher  knowledge  and  on  the  content  and  domains  of 
teacher  knowledge  support  the  idea  that  teachers’  knowledge  and  the  way  in  which  that  knowledge  is  organized  of  crucial 
influence  on  teacher  thinking  and  action. 

The  advocates  of  the  epistemological  perspective,  on  the  other  hand,  believe  that  most  of  teacher  knowledge  will 
not  be  organized  as  knowledge,  rather  such  knowledge  is  seen  as  persona)  and  practical  knowledge  (Clandinin  & 

Connelly,  1987)  which  is  not  solely  cognitive  in  character.  From  this  perspective,  influence  of  the  wider  sociocultural 
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system,  coupled  with  the  influence  of  the  immediate  learning  environment,  have  the  most  contrasting  effect  on  teachers' 
knowledge.  Together  with  this  wide  concept  of  knowledge,  the  relationship  between  theory  and  practice  has  changed. 
Scientifically  based  knowledge  is  no  longer  considered  the  most  important  knowledge  for  teachers.  Instead,  the 
importance  of  experienced-based  knowledge  has  been  emphasized.  This  implies  a transition  from  studying  the 
relationship  between  teacher  thinking  and  actions  to  studying  teachers'  actions  as  an  expression  of  teacher  thinking. 

Recent  research  on  teacher  thinking  vie'‘  teacher  knowledge  as  an  interpretive  framework,  or  series  of  "implicit 
theories,"  by  which  teachers  attach  meaning  to  their  environment  and  guide  their  actions  within  it  (Clark,  1988; 
Calderhead,  1987;  Zeichner,  Tabachnick,  & Densmore  1987). 

The  above  concept  of  teacher  knowledge  used  in  recent  research  on  teacher  thinking  is  most  developed  by 
Schon's  epistemology  of  practice.  He  uses  the  term  "knowledge  in  action"  or  "reflection-in-action"  to  express  the 
relation  between  theory  and  practice.  The  reflection-in-action  epistemology  suggests  that  teachers  address  problematic 
situations  by  recalling  elements  of  similar  past  situations,  selecting  a move  derived  from  a tentative  interpretation  of  the 
present  situation,  attending  to  the  "back  talk"  in  reaction  to  the  move  and  reframing  or  reinterpreting  the  situation 
(Schon,  1987).  Schon's  work  has  been  very  influential  in  validating  a focus  on  the  practical  knowledge  of  teachers. 

The  current  research  on  teacher  thinking  derived  from  Schon’s  epistemology  of  knowledge-in-action  further 
emphasizes  personal  and  practical  knowledge  (or  pedagogical  content  knowledge)  within  a broader  range  of  teachers’ 
"professional  knowledge".  Teachers'  professional  knowledge  goes  beyond  the  classroom  realities.  It  consists  of 
knowledge  of  socio-political  and  contextual  realities  of  school  life.  It  defines  teacher’s  personal  and  practical  knowledge 
within  a broader  institutional  context  (Goodson,  & Cole,  1994). 

The  Influence  of  Context 

With  the  shift  of  metaphor  from  teachers  as  decision  makers  to  teachers  as  sense  makers  and  reflective 
practitioners,  the  dominant  psychological  epistemology  of  individual  cognitive  processing  and  technical  action  or 
knowledge  being  solely  in  the  mind  of  the  individual  has  been  questioned  (Gergen,  1985;  Yinger  & Hendricks-Lee, 

1993).  The  immersion  theory  of  situated  cognition  has  changed  the  conception  of  thinking  as  a process  within  an 
individual’s  mind,  perhaps  influenced  by  a context  provided  by  the  situation,  to  the  conception  of  thinking  as  an 
interaction  between  an  individual  and  a physical  and  social  situation  (Greeno,  1989).  Based  on  this  social  cognitive 
perspective,  context  influences  knowledge  structures,  as  well  as  beliefs  and  theories.  Due  to  the  situational  aspects  of  the 
settings,  a knowledge  structure  elicited  from  a teacher  in  one  setting  will  not  necessarily  be  the  knowledge  structure 
elicited  in  another  setting  (Roehler,  Duffy,  Herrmann,  Conley,  & Johnson,  1988).  Teachers'  professional  behavior, 
therefore,  always  conforms  to  the  structural,  cultural  and  organizational  context  in  which  they  work. 

This  view  of  knowledge  shifted  the  focus  of  research  on  teacher  thinking  from  the  individual  teacher’s  cognitive 
processing  and  technical  action  to  the  relationships  between  the  individual  and  the  context.  As  indicated  earlier,  what  is 
learned  by  teachers  is  thought  to  depend  primarily  on  what  is  provided,  and  how  it  is  provided  by  the  wider  sociocultural 
system  and  the  immediate  learning  environment.  Knowledge  is  always  marked  by  its  situations-past  and  present  use 
(Cazden,  1989).  Moreover,  the  situations  of  classroom  and  school  life  do  not  produce  the  same  responses  in  every 
teacher.  The  background  differences  that  teachers  bring  to  their  classroom  and  school  lead  them  to  generate  and  organize 
knowledge  in  very  different  ways. 

A more  recent  focus  on  teacher’  lives  and  personal  biographies  consequently  has  conceptualized  teacher 
development  as  rooted  in  the  personal  experiences  (e.g.,  Bullough,  Knowles  & Crow,  1992;  Butt  & Raymond,  1987; 
Clandinin,  1986;  Connelly  & Clandinin,  1990;  Knowles,  1992;  Knowles  & Holt-Reynolds,  1991).  Other  studies  have 
argued  for  the  personal  mode  as  being  linked  to  broader  contextual  parameters  (e.g.,  Apple,  1986;  Ball  & Goodson, 

1985;  Britzman,  1986;  Cole,  1990,  1991;  Fox,  1992;  Goodson  & Cole,  1994;  Goodson,  1989,  1988,  1989,  1990; 
Kelchtermans,  1993;  Laffite,  1993;  Zeichner  & Grant,  1981;  Zeichner  & Tabachnick,  1985).  This  latter  view  stresses 
that  teachers'  professional  behavior  always  takes  place  within  the  institutional  context  of  a school.  The  teacher  is  a 
member  of  a team  and  is  held  accountable  for  his/her  job  by  the  principal  and  external  bodies  (Kelchtermans,  1993).  It  is 
true  that  there  is  a range  of  personal,  practical  and  pedagogical  knowledge  that  is  important  in  understanding  the  teacher’s 
conduct  in  classroom.  There  is  also  a range  of  knowledge  of  great  importance  that  deals  with  the  contextual  realities  of 
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school  life.  This  is  critically  important  because  it  affects  the  lives  and  arenas  in  which  personal,  practical,  and 
pedagogical  knowledge  are  utilized  and  applied  (Goodson  & Cole,  1994).  Events  and  experiences,  both  past  and  present, 
that  take  place  at  home,  school  and  in  broader  social  spheres  help  shape  teachers'  lives  and  careers. 

Culture  structures  are  general  guidelines  by  which  people  ought  to  live.  These  general  guidelines  affect 
individuals'  behaviors,  their  interactions  with  family  and  friends  and  even  their  approaches  to  education  (Hamilton,  1993). 
The  importance  of  social  contexts  (Dewy,  1904)  cultural  settings  (Philips,  1983)  and  social  construction  (Berger  & 
Luckman,  1966)  in  schools  has  been  recognized  for  some  time,  and  much  research  since  that  time  has  supported  the 
importance  and  impact  of  the  social  contexts,  cultural  settings  and  social  construction  in  the  school  arena.  The  culture  of 
school  shapes  teachers'  understanding  of  their  own  actions  (Hamilton,  1993;  Feiman-Nemser  & Floden,  1986;  Lortie, 
1975;  Sarason,  1982;  Metrz,  1983,  1986;  Rutter,  Maughan,  Mortimore  & Oston,  1979)  and  that  of  their  students  (Tyler, 
1987)  and  is  grounded  in  faculty  members'  shared  definitions  (Page,  1988).  Such  understanding  is  also  linked  to  a larger 
social  class  of  the  school  and  to  the  culture  of  the  community. 

Much  research  on  teacher  thinking  within  the  paradigm  of  interpretive  research  has  examined  the  connections 
between  teacher  knowledge  and  beliefs  and  their  culture.  Olson  (1988)  contends  that ".  . . . what  teachers  tell  us  about 
their  practice  is,  most  fundamentally,  a reflection  of  their  culture  and  cannot  be  properly  understood  without  reference  to 
that  culture.  . . (1988,  p.  69).  Cultural  knowledge  has  been  studied  by  looking  at  cultural  patterns  in  teachers'  life 
histories  (e.g.,  Butt,  1984;  Clandinin  & Connelly,  1987;  Elbaz,  1983,  1990;  Richardson,  1989;  Tobin,  1990;  Wallace  & 
Louden,  1992)  and  critical  incidents  in  teachers'  lives  (e.g.,  Sikes,  et  al.,  1985;  Measor,  1985). 

Thus  a shift  has  emerged  recently  toward  describing  teachers'  knowledge  and  practice,  not  only  in  the  classroom 
context,  but  also  within  the  framework  that  recognizes  the  influence  of  social  and  institutional  cultures.  Finding  ways  to 
connect  teacher  thinking  and  action  with  the  context  is  the  focus  of  this  line  of  research  on  teacher  thinking. 

Toward  a New  Conceptual  Model  of  Thinking  about  Teaching 

Although  research  on  teachers'  thinking  does  not  provide  us  with  a comprehensive  theoretical  framework  for 
thinking  about  teaching,  a number  of  assumptions  can  be  derived  from  research  and  theory  in  this  field.  These 
assumptions  combined  with  the  theories  and  principles  of  instructional  design  models  provide  a holistic  picture  that 
enables  us  to  conceptualize  the  teachers'  performance  in  the  context  of  the  classroom.  The  scheme  proposed  here  (see 
Figure  1)  provides  a framework  for  thinking  about  teacher's,  teaching  or  instruction.  The  scheme  explains  the  major 
factors  in  teachers’  thinking  processes  and  assumed  relationships  among  them. 


Learning  Context 


Teaching  Context 


Students'  Reactions 
and  Perspectives 


Figure  1 . Conceptual  Framework  for  Research  on  Teaching  and 
Instructional  Design 


Three  (3)  basic  characteristics  define  the  proposed  scheme.  First,  it  proposes  that  teaching  is  a complex,  highly 
contextualized  profession,  involving  a complex  form  of  social  interaction  that  varies  depending  on  context  (Calderhead, 
1987;  Clark,  1986;  Clark  & Lampert,  1986).  The  relationship  between  teacher's  thought  and  his/her  action  can  only  be 
described  in  relation  to  a specific  learning  and  teaching  context.  By  learning  context  we  mean  an  environment  where 
teachers,  as  learners,  are  part  of  a community  that  actively  work  together,  and  share  the  same  social  and  organizational 
culture  (Britzman,  1986,  1991;  Fox,  1992;  Goodson  & Cole,  1994).  Such  context  affects  teacher  knowledge  and 
professional  responsibilities  by  formal  structure  and  pressures  they  exact  on  the  teacher  (Clark,  1986;  Carter,  1990;  Fox, 
1992;  Gudmundsdottir,  1991;  Laffitte,  1993;  Pope,  1993;  Yinger,  1993).  By  teaching  context  we  mean  teachers,  as 
learners,  think  on  the  spot,  in  relation  to  the  context  of  the  problem  and  the  social  climate  of  the  classroom.  Subject 
matter  proficiency  is  not  enough;  teachers  also  need  to  see  the  subject  in  the  way  their  students  see  it  in  order  to 
transform  that  vision  to  benefit  their  students.  Hence,  what  teachers  do  can  never  be  comprehended  solely  in  terms  of 
teaching  and  learning  academic  subject  matter.  The  formal  curriculum  of  academic  knowledge  and  skills  has  a counterpart 
"hidden  curriculum"  of  values  and  behavior,  which  is  embedded  in  the  social  and  cultural  systems  of  the  school  and 
classroom. 

Second,  teacher  knowledge  is  grounded  in  a systemic  and  holistic  notion  of  learning,  knowledge,  practice  and 
relationships.  It  is  constructed  in  the  process  of  reflection,  inquiry  and  action  by  teachers  themselves  (Schon,  1987),  and 
it  is  used  in  complex  ways  during  the  process  of  planning  for  and  executing  teaching  activities,  as  well  as  in  making 
sense  of  decisions  already  made  (Johnston,  1992).  Knowledge  consists  of  past  construction  (Fosnot,  1989).  Teachers 
bring  a set  of  predisposed  and  personal  knowledge  from  their  private  lives  to  their  practice  of  teaching  (Johnson,  1989; 
Johnston,  1992). 
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Third,  teaching  is  a dynamic,  on-going,  social  and  dialectical  process  which  is  intended  to  change  (Calderhead, 
1987).  Change  in  the  outer  layers  (teachers’  knowledge  and  beliefs)  is  basically  caused  by  the  center  of  the  cycle 
(teachers’  reflective  action),  although  the  factors  coming  from  one's  interpretation  of  the  context  can  also  cause  changes  in 
the  outer  layers.  Teachers'  conceptions  of  the  context  in  their  professional  world  are  important  to  consider  when  they  try 
to  change  their  teaching  practice  (Carlgren,  Lindblad,  1991).  The  concept  of  the  teacher’s  culture  as  a creative,  historical 
system  of  symbols  and  meanings  is  also  central  to  this  change  (Brown,  et  al.  1989;  Yinger,  1990).  The  concept  of 
culture  is  not  limited  only  to  verbal  expression.  Both  verbal  and  nonverbal  communication  are  culturally  patterned  even 
though  the  teacher  may  not  be  aware  of  it.  The  meaning  the  teacher  gives  to  his/her  experiences  (actions)  differs  from 
culture  to  culture.  In  other  words,  the  way  a teacher  of  a particular  culture  (e.g.,  gender,  ethnicity,  class)  categorizes  and 
interprets  his/her  actions  may  be  different  from  the  way  another  teacher  from  another  culture  would  view  them. 

Figure  1 also  shows  the  basic  elements  of  teaching  and  teachers'  cognition.  According  to  this  framework,  in 
order  to  explain  teachers'  behavior,  it  is  necessary  to  look  at  the  classroom  as  a social  and  cultural  system  characterized  by 
reciprocity  among  participants  and  between  the  participants  and  the  physical  setting.  The  basic  elements  of  teaching  are 
the  teachers'  knowledge,  beliefs  and  perceptions  of  their  craft,  which  incorporates  various  students'  reactions  and 
perspectives.  Skill  in  teaching  rests  on  teachers'  knowledge,  which  can  be  divided  into  pedagogical  content  knowledge, 
personal  knowledge,  knowledge  of  curriculum  and  knowledge  about  learners  (Clandinin,  1986;  Shulman,  1986a). 

Teachers'  knowledge  and  beliefs  move  in  a spiral  toward  more  concrete  levels  of  knowledge,  which  are  prediction,  action, 
and  reflection.  There  is  a direct  relationship  between  teachers’  knowledge  and  beliefs,  and  the  way  they  interpret  their 
practice,  although  much  of  the  knowledge  that  teachers  hold  and  act  on  is  tacit.  The  beliefs,  values  and  norms  that 
teachers  come  to  have  the  most  faith  in  and  use  most  frequently  to  guide  their  practice  are  those  consistent  with 
predictions  that  have  ’’ worked1'  in  the  complex  and  demanding  classroom  arena  (Clark  & Yinger,  1987).  While  teachers' 
preactive  plans  (predictions)  provide  frameworks  for  what  is  possible  or  even  likely  to  occur  in  classrooms  in  practice, 
these  frameworks  do  not  function  as  rigid  scripts  for  teacher  activity.  Instead,  in  the  interactive  process  of  teaching  in 
which  cognition  is  translated  into  action,  teachers  engage  in  moment-to-moment  decision  making  and  problem  solving 
based  on  their  perceptions,  practical  knowledge  and  judgments  about  the  events.  Teachers’  metacognitive,  purpose-driven 
behavior  and/or  reflection  on  the  effect  of  an  action  helps  them  modify  their  previous  pedagogical  concepts  or  build  a new 
pedagogical  principle  which,  in  turn,  affects  their  future  thinking,  planning  and  action. 

Implications  of  the  above  Conceptual  Model  for  Research  and  Instructional  Design  Practice 

The  above  conceptual  model  proposes  several  changes  in  the  ways  that  research  has  been  conducted  in  this  area. 
First  it  suggests  that  teachers  and  their  teaching  and  learning  processes  can  only  be  studied  within  their  social  and  cultural 
context.  To  understand  why  teachers  do  what  they  do,  researchers  need  not  only  to  focus  on  the  present  social  context  of 
teaching  but  they  also  need  to  consider  the  historical  and  organizational  context  in  which  teachers  have  worked  for  years. 
Within  the  historical  context  of  teaching,  teachers’  life  history,  as  a social  context  of  their  beliefs  and  culture  is  of 
particular  importance. 

The  above  conceptual  framework  also  suggests  that  teachers'  knowledge  is  a complex  blend  of  personal,  practical 
and  theoretical  knowledge  . What  teachers  do  is  a reflection  of  this  complex  and  highly  social  organization  of  knowledge 
and  cannot  be  studied  without  gaining  access  to  them.  Research  in  teaching,  learning  and  instruction,  therefore,  has  to 
shift  its  emphasis  from  cognition  to  social  construction  of  knowing. 

The  above  model  also  suggests  some  changes  in  instructional  design  models  and  principles.  First,  the  image  of 
teachers  as  designers  of  their  own  instruction  needs  to  be  emphasized  in  the  instructional  technology  field.  However,  the 
instructional  design  models  and  principles  should  be  reconceptualized  if  they  are  to  be  used  by  teachers.  The  instructional 
design  models  and  principles  are  to  consider  the  social  context  of  learning  and  teaching.  The  new  conceptualization  of  the 
design  processes  should  focus  on  the  decision  making  processes  that  can  describe  the  complex,  uncertain  and  content  and 
context-bound  approach  to  instructional  design.  This  new  conceptualization  should  then  consider  several  issues  with 
regard  to  the  concept  of  design  in  real-world  settings:  the  context  or  environment  of  the  design,  the  experience  of  the 
designer  and  his/her  frame  of  reference,  the  content  specific  characteristic  of  the  design,  and  finally  the  interactive  and 


dynamic  nature  of  the  design  activity.  Each  one  of  these  issues  should  be  further  investigated  and  incorporated  in  the 
conceptual  theory  of  instructional  design. 

Second,  the  concept  of  design  as  an  artistic,  social  and  cooperative  act  should  replace  the  procedural  and  technical 
concept  of  design.  Instructional  design  activities  should  be  focused  on  the  product  of  the  design  instead  of  focusing  on 
the  procedure.  Once  the  design  is  conceptualized  as  an  integrated  and  systemic  (holistic)  approach  that  can  only  be 
improved  through  constant  and  continuous  reflection,  different  instructional  design  techniques  or  procedures  (analysis, 
design,  development  and  implementation  and  evaluation)  can  be  seen  or  isolated  as  significant  skills  to  improve  teaching 
performance  or  developed  instructional  product.  An  integrated  and  systemic  approach  to  instructional  designs  allows  the 
teacher  designer  flexibility  of  the  design  and  emphasizes  the  teacher  designer’s  interpretations  of  his/her  performance  with 
respect  to  instructional  context. 

Finally,  instructional  design  models  and  principles  should  focus  on  an  approach  in  which  the  design  objectives 
and  strategies  or  solutions  evolve  as  the  teacher  designer  becomes  more  acquainted  with  the  social  and  cultural  system  and 
subsystems.  It  should  also  take  an  approach  in  which  people  who  are  affected  by  the  design,  including  learners,  are 
actively  participating  in  the  decision  making  process.  The  participatory,  dialectic  and  evolving  design  approach  that 
focuses  on  the  process  of  negotiation  is  a form  of  design  that  integrates  the  four  familiar  phases  (analysis,  design  and 
development,  implementation  and  evaluation)  of  instructional  design  models  into  one  activity.  It  seems  that  a cognitive, 
objective  and  systematic  (step-by-step)  approach  to  designing  instruction  should  be  reconsidered  if  instructional  design 
models  are  to  be  used  by  teachers. 
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Introduction 


For  decades,  skills  in  instructional  design  and  technology  have  not  been  tapped  by  public  schools.  One  resource 
that  has  been  consistently  overlooked  in  public  school  systems  is  the  pool  of  educators  who  have  been  trained  in 
instructional  design  and  technology.  Even  in  the  educational  technology  field,  little  attention  is  given  to  the  role  that 
instructional  designers/technologists  can  play  in  solving  the  problems  facing  the  schools,  including  integration  of 
technology  into  classroom  practices.  Instead,  the  emphasis  has  been  on  increasing  the  extent  to  which  computers  are 
used  for  instructional  purposes.  It  is  optimistic  to  assume  that  the  increased  use  of  computers  in  schools  will  necessarily 
result  in  properly  integrating  technology  into  instruction  to  improve  students'  learning  outcomes.  Professionals  in  the 
field  agree  that  the  value  of  learning  technology  is  actually  dependent  upon  what  use  we  make  of  them. 

Leaders  in  the  field  of  instructional  technology  seem  to  loose  hope  of  getting  instructional 
designers/technologists  into  school  systems  in  order  to  help  teachers  to  properly  design  their  curriculum  while  integrating 
new  technology  (Reiser,  1988).  Hence,  most  professionals  in  the  field  focus  their  attention  on  business  and  industry. 
Others  have  tried  to  influence  classroom  instruction  by  developing  instructionally  designed  software  for  classroom 
instruction  and/or  developing  automated  instructional  design  software  for  teachers  (Merrill  & Li,  1988,  1989).  Still 
others  place  greater  emphasis  on  preservice  teacher  education  programs  to  prepare  teachers  to  learn  not  only  how  to  use 
technology,  but  also  how  to  integrate  it  into  their  instruction  (e.g.,  Dick  & Reiser,  1989;  Reiser,  1988;  Reiser  & Dick, 
1996;  Rosset  & Garbosky,  1987;  Sullivan  & Higgins,  1983;  Zellner  & Rieber,  1991).  There  has  also  been  strong 
support  for  school  restructuring  initiatives  among  leaders  of  the  field.  Proponents  of  systemic  change  in  education 
believe  that  the  existing  system  is  outdated  and  is  no  longer  serving  the  needs  of  learners  (Mory  & Salisbury,  1992; 
Reigeluth,  1992;  Salisbury,  1993).  Therefore,  they  suggest  to  replace  the  existing  system  with  one* which  is  more 
appropriate  to  the  needs  of  students  in  the  twenty-first  century. 

While  educational  technologists  are  trying  to  find  ways  of  becoming  more  involved  in  solving  educational 
problems,  school  systems,  under  pressure  for  reform  and  accountability,  have  begun  to  use  their  technological  resources 
to  ensure  increased  academic  outcomes  of  today's  diverse  student  population.  Lamenting  that  previous  educational  media 
such  as  language  arts  labs,  closed  circuit  television,  and  movie  projectors  yielded  less  than  the  anticipated  results  in 
public  schools  and  that  microcomputers  are  almost  meeting  a similar  fate,  it  seems  that  school  systems  are  beginning  to 
realize  that  the  problem  is  more  instructional  rather  than  technological.  Emphasis  of  many  school  reform  initiative  plans 
for  the  integration  of  computers  into  classrooms  across  the  curriculum  (e.g..  Levy,  1986;  Mergendoller  & Pardo,  1991; 
Sanacore  & Alio,  1989)  is  one  example  of  these  efforts.  Another  example  is  public  schools’  attempt  to  establish  a 
technical  and  instructional  support  system  at  the  school  level  to  assure  the  proper  usage  of  the  technology,  mainly 
computers.  This  attempt  resulted  in  the  creation  of  a professional  job  position  that  seems  to  go  beyond  utilization  of 
media  and  library  resources.  The  fact  that  a technology  coordinator  or  the  Technology  Resource  Teacher  is  assigned  to 
this  position  in  several  states,  districts,  and/or  counties  demonstrates  public  schools’  desire  to  focus  on  integration  of 
technology  into  classroom  instruction.  But,  how  successful  they  are  and  whether  or  not  they  are  achieving  what  they 
desire  to  achieve  has  not  been  studied. 

It  appears  that  the  instructional  designers'  expertise  is  needed  more  now  in  the  public  schools  than  any  time 
before.  It  is  obvious  that  if  schools  are  to  look  beyond  the  number  of  computers  and  their  usage,  an  instructional 
support  system  is  needed  to  help  teachers  design  their  instruction  in  a way  that  new  technology  is  an  integral  part  of  it. 
The  questions  are  raised:  Do  the  school  systems  consider  or  mention  instructional  designers/technologists  as  a solution  to 
their  problems?  Will  they  ask  for  the  instructional  designers/technologists'  help  in  the  future?  Experience  shows  that 
the  answer  to  this  question  is  "No".  However,  very  few  researchers  have  tried  to  empirically  find  those  conditions  under 
which  instructional  designers  or  technologists  may  be  best  involved  in  the  integration  of  technology  into  classroom 
practices.  It  is  not  hard  to  anticipate  that,  given  emerging  technologies  and  the  increasing  demands  and  needs  for 
integration  of  these  technologies  to  the  classrooms,  the  field  of  instructional  technology  can  provide  the  best 
instructional  support  for  teachers  and  the  means  by  which  to  keep  them  updated. 
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Th:  Purpose  of  the  Study 


In  an  attempt  to  respond  to  school  reform  initiatives  and  a State-mandated  use  of  computers  in  the  public  school 
classroom,  one  school  district  located  in  southeastern  North  Carolina  began  to  implement  a computer  competency 
program  in  the  local  elementary,  middle,  and  high  schools  beginning  in  September,  1994.  This  program  was  to  be 
integrated  into  the  classroom  curriculum  in  such  a way  that  the  focus  was  not  merely  on  the  operational  use  of 
technology,  but  on  the  use  of  technology  as  a tool  to  facilitate  students'  learning  by  using  technology  to  generate  ideas 
and  access  information.  To  reach  this  goal,  the  teachers  are  expected  to  design  and  implement  instruction  in  a way  that 
technology  is  a part  of  their  classroom  practice.  Thus,  teachers  need  not  only  to  have  skills  in  operating  and  using 
technology,  but  also  to  have  skills  in  designing,  developing,  implementing  and  evaluating  curriculum  and  instruction 
which  integrate  different  technologies  and  delivery  systems. 

Most  teachers  in  the  county  do  not  possess  skills  to  use  technology  in  their  classrooms,  nor  do  they  know  how 
to  integrate  the  technology  into  their  instructional  practices.  To  begin  this  educational  process,  the  school  district  has 
recently  hired  thirteen  Technology  Resource  Teachers  to  support  and  facilitate  integration  of  technology  into  classroom 
teachers'  instructional  practices.  Some  of  the  resource  teachers  have  been  in  their  schools  for  several  years,  and  were 
reappointed  within  their  school  to  be  full-time  technology  educators. 

The  purpose  of  this  study  was  to  explore  the  roles  and  responsibilities  of  these  Technology  Resource  Teachers 
(“TRT”)  and  the  effects  that  their  roles  have  on  the  integration  of  technology  in  classroom  instruction.  The  study' also 
sought  to  examine:  (i)  how  Technology  Resource  Teachers'  roles  differ  from  those  of  instructional  designers  and 
technologists  if  they  were  to  be  involved  in  public  schools,  and  (ii)  what  an  instructional  technologist  might  do  if  he  or 
she  was  assigned  to  this  position.  The  study,  therefore,  focused  on  the  following  specific  questions: 

• What  are  the  roles  and  responsibilities  of  these  educators? 

• What  role  and  expectations  does  the  district  have  for  these  educators  and  how  are  these  roles  and  expectations 
being  fulfilled? 

• Do  the  resource  teachers  possess  the  skills  and  knowledge  required  to  successfully  play  their  role?  If  so,  how  did 
they  acquire  those  skills  and  knowledge? 

• How  do  the  TRTs  affect  integration  of  technology  into  classroom  practice? 

• What  kind  of  skills  and  knowledge  does  a teacher  have  to  possess  to  be  eligible  for  being  a TRT? 

• What  are  the  differences  and  similarities  between  the  TRT  and  an  instructional  designer/technologist? 

• Would  the  role  of  the  TRT  propose  a new  set  of  competencies  for  instructional  designers/  technologists  who 
want  to  serve  the  school  system? 

• How  are  these  competencies  different  from  those  of  a media  specialist? 

The  Research  Method  and  Its  Framework 


A naturalistic  and  participant-oriented  model  within  the  paradigm  of  naturalistic  studies  was  used  for  conducting 
this  exploratory  study.  This  approach  was  perceived  to  be  appropriate  because  it  could  help  researchers  describe  the 
program  holistically  and  understand  it  from  the  perspective  of  participants.  It  also  provided  enough  flexibility  to  base 
information  on  program  activities  rather  than  on  program  intent.  It  also  made  it  possible  to  employ  multiple  data- 
gathering  methods,  especially  observation  and  interviews.  Rippey  (1973)  suggests  this  approach  for  understanding  the 
process  of  change  in  various  types  of  organizations. 

The  following  data-collection  methods  were  used  in  this  study: 

• informal  and  formal  interviews, 

• classroom,  workshop,  and  site  observations, 

• public  and  personal  records,  reports  and  documents, 

• questionnaire  and  attitude  surveys,  and 

• focus  group  discussion. 
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Due  to  the  limitation  of  time  and  resources  and  because  the  program  was  first  conducted  at  the  middle  school 
level,  the  present  study  focused  only  on  middle  schools  in  the  county  (six  in  total).  The  data  collection  began  in  early 
January,  1995  and  continued  until  early  December,  1995.  From  January  to  March,  1995  the  study  focused  on  gathering 
preliminary  data  about  the  program,  conducting  interviews  with  the  Technology  Coordinators  for  the  county,  reviewing 
documentation  on  the  roles  and  responsibilities  of  the  TRTs,  as  well  as  documentation  on  the  whole  program,  and 
visiting  each  of  the  sites. 

From  March  to  May,  1995,  interviews  were  conducted  with  the  TRTs,  their  classroom  and  training  workshops 
were  observed,  and  attitude  and  general  information  surveys  were  administered  to  teachers  on  each  of  the  sites  being 
observed.  The  general  survey  was  constructed  by  the  researchers  around  the  major  questions  of  the  study.  A Computer 
Literacy  Attitude  Survey  (“CLA”)  was  also  adopted  and  completed  by  the  teachers  at  each  site.  (Both  of  these  surveys 
may  be  viewed  in  Appendix  A.)  Both  survey  instruments  were  completed  by  teachers  following  one  of  the  school’s 
professional  development  meetings  upon  the  request  of  the  respective  Technology  Resource  Teacher. 

The  General  Information  Questionnaire  consisted  of  1 8 closed-ended  and  nine  open-ended  items.  The  closed-ended 
items  asked  about  teachers'  computer  facilities  at  home  and  the  classroom,  their  usage  of  their  computer  facilities,  their 
previous  and  present  computer  training,  the  computer  competency  tests,  and  their  feelings  about  their  computer  skills. 
The  open-ended  items,  on  the  other  hand,  asked  teachers  about  their  perception  of  the  role  and  responsibilities  of  the 
TRTs,  and  the  nature  of  the  help  that  teachers  receive  from  the  TRT  in  their  respective  school.  The  CLA  survey 
consisted  of  50  Likert-scale  type  items  which  were  developed  and  used  by  Savenye  and  her  colleagues  (Savenye,  1992; 
Savenye,  Davidson,  & Orr,  1992)  to  measure  attitudes  of  preservice  teachers  toward  computers.  The  items  were  slightly 
modified  to  be  used  with  in-service  teachers.  In  addition  to  the  Likert-scale  items,  teachers  were  asked  several  questions 
about  their  background  and  the  number  of  hours  that  they  had  participated  in  computer  training  workshops.  The  return 
rate  for  both  the  general  survey  and  the  CLA  survey  was  between  90-95%  for  two  of  the  schools  and  was  between  55- 
71%  for  the  other  four  schools. 

During  the  1995  fall  semester,  interviews  were  also  conducted  with  the  principals  of  the  schools,  and  a focus 
group  discussion  was  conducted  with  the  technology  coordinators  of  the  county.  All  of  the  interviews  and  focus  group 
discussions  were  audio-taped  and  transcribed  for  further  analysis.  Field  notes  were  used  for  recording  observation  data  and 
informal  interviews. 


DataAnalysis 


Data  were  analyzed  using  both  qualitative  and  quantitative  analysis  techniques.  Interview  and  observation  data, 
the  results  of  the  open-ended  questions  from  the  general  survey,  and  the  collected  documents  were  analyzed  qualitatively 
using  the  Miles  and  Huberman  (1984)  model.  Based  on  this  approach,  the  first  part  of  the  analysis  was  data  reduction. 
During  this  process,  the  data  chunks  were  identified  and  coded,  the  patterns  that  best  summarized  a number  of  chunks 
were  sorted  and  then  were  further  subsumed  into  larger  patterns.  In  some  cases,  the  data  were  organized  using  the 
frequency  of  the  responses  to  specific  questions  or  by  the  pattern  of  responses.  In  such  cases,  however,  the  numbers  were 
used  together  with  the  words  to  keep  the  data  in  its  context.  During  the  second  analytical  stage,  the  data  were 
summarized  and  organized  using  matrices,  charts  and  tables.  This  stage  helped  the  researchers  interact  with  the  data  and 
draw  their  preliminary  conclusions,  which  in  turn  went  through  another  round  of  testing  and  verification  using  all 
different  sources  of  the  data  for  the  final  conclusion. 

The  results  from  the  closed-ended  items  of  the  general  survey  and  the  attitude  questionnaire  were  analyzed 
quantitatively.  Descriptive  statistics,  cross  tabulation,  Pearson  correlation,  and  MANOVA  were  the  quantitative 
techniques  that  were  used  for  this  part  of  the  data. 
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Results 


Description  of  the  School  District  and  the  Technology  Initiatives 

The  study  was  conducted  in  all  of  the  middle  schools  (a  total  of  six)  within  a city  district  in  the  northeastern 
United  States.  The  district  is  within  a zone  that  draws  from  affluent  neighborhoods,  as  well  as  from  neighborhoods  of 
racially-  and  ethnically-mixed  working-  and  low-class  families. 

In  1992,  the  State  of  North  Carolina  Department  of  Public  Instruction  put  together  a set  of  computer  skill 
competencies  as  a basic  requirement  of  public  school  students  and  teachers.  In  anticipation  of  this  state  technology 
initiative  that  requires  all  schools  to  integrate  technology  into  their  classrooms  and  all  teachers  and  students  to  pass  a 
technology  competency  test,  the  district  technology  coordinators  had  to  design  a plan  that  would  enable  the  classroom 
teachers  in  the  district  to  begin  to  attain  these  computer  skills.  While  the  elementary  schools  in  the  district  already  had  a 
designated  classroom  teacher  to  act  as  a technology  resource  person,  and  the  classroom  teachers  were  already  using 
technology  in  the  elementary  classrooms  to  some  degree,  the  middle  schools  in  the  district  were  not  set  up  with  such  a 
situation.  Since  the  mandate  included  a timeline  to  eventually  administer  a test  to  all  eighth  graders  in  the  state,  insuring 
that  students  had  met  the  minimum  computer  skills,  it  was  imperative  that  the  middle  school  teachers  and  students  be 
targeted  for  broader  technology  support. 

The  TRT  position  was  created  to  provide  technical  and  instructional  support  at  the  school  level  to  help  teachers 
integrate  technology  into  their  curriculum  and  classroom  practices,  and  to  assist  teachers  in  passing  the  technology 
competency  test  required  by  the  state.  This  position  was  created  to  be  different  from  the  media  specialist  position  which 
already  existed  in  most  schools.  The  role  of  the  TRT  differs  from  that  of  the  media  specialist  because  the  TRT  was 
expected  to  be  responsible  for  integration  of  computer  technology  into  classrooms,  while  the  media  specialist  was 
responsible  for  library  media  and  providing  print  and  media  support  at  the  school  library.  Two  technology  supervisors  at 
the  district  level  were  in  charge  of  developing  a job  description  for  the  TRT  position  and  helping  principals  hire  the 
qualified  individuals.  These  two  supervisors  were  also  in  charge  of  preparing  the  TRTs  for  their  job  responsibilities  by 
providing  them  with  training  workshops  and  proper  technical  and  administrative  support. 

The  Technology  Resource  Teachers:  Who  They  Are  and  What  They  Do? 

Six  technology  resource  teachers  were  interviewed  in  this  study.  Their  educational  background  spans  from  one 
year  of  previous  teaching  experience  to  twenty-two  years  of  previous  teaching  experience.  The  majority  of  them  have  a 
background  in  teaching  math  (83%)  and/or  science  (16.7%).  Four  of  the  TRTs  were  classroom  teachers  in  their  respective 
schools  before  being  appointed  to  this  position.  Two  of  the  TRTs  were  hired  from  outside  the  school  and  were  new  to 
the  school  as  well  as  to  the  appointed  position.  In  general,  the  TRTs’  background  in  computer  training  incorporates 
college  courses  and  in-service  training  workshops  (50%),  job  related  experiences  (33.3%)  and  self-leaming/practices 
( 1 6.7%).  Each  of  the  TRTs  have  passed  the  computer  competency  test  required  by  the  state.  Their  reasons  for  becoming 
TRTs  include  an  interest  in  technology  and  an  interest  in  teaching.  None  of  the  TRTs,  however,  have  had  any  training  or 
college  courses  in  instructional  design  and  technology.  They  also  have  not  had  any  training  and/or  experiences  in 
analysis,  design,  development  and/or  evaluation  of  instructional  materials  and/or  programs. 

The  observation,  interview  and  questionnaire  data  show  that  all  six  TRTs  have  formed  the  computer  lab  at  their 
school  and  have  maintained  its  operation  throughout  the  year.  They  have  used  the  computer  lab  to  teach  technology  skills 
to  the  teachers  and  to  the  students  in  their  schools  . The  main  focus  areas  in  workshops  are:  database,  word  processing 
and  spreadsheets.  Terms  and  operation,  keyboarding,  societal  uses,  and  ethics  have  also  been  taught.  Teachers  at  each 
school  then  were  given  the  choice  of  attending  the  training  workshops  provided  at  their  site  or  other  schools.  Since  the 
response  rate  was  not  equal  for  each  school,  a random  sample  of  17  responses  was  selected  from  each  school  for  further 
analysis.  A chi-square  test  of  difference  between  the  number  of  training  workshops  provided  by  each  school  (after 
equalizing  the  sample  size  across  the  schools)  shows  that  teachers  in  each  school  have  received  equal  number  of  training 
workshops  (see  Table  1).  This  result,  combined  with  the  observation  and  interview  data,  indicates  that  all  six  TRTs 
provided  similar  houis  of  training  workshops  for  teachers  in  their  respective  schools. 


However,  the  analysis  of  data  from  different  sources,  including  the  general  survey,  suggests  that  more  teachers  in 
School  2 were  able  to  take  the  computer  competency  test  in  all  five  areas  (see  Table  1).  School  1 rates  second  with 
respect  to  the  number  of  teachers  who  were  able  to  take  competency  tests  in  all  areas,  except  for  keyboarding.  Data 
shows  that  the  teachers  in  School  2 and  School  1 do  not  rate  their  computer  skills  significantly  higher  than  the  other 
schools.  In  some  areas,  such  as  word-processing,  keyboarding  and  spreadsheet.  School  2 and  School  1,  in  fact,  rate  their 
computer  skills  lower  than  School  3 (see  Table  2).  Also,  the  teachers’  previous  experiences  with  computers  in  School  1 
and  School  2 are  not  significantly  different  from  the  other  schools  (see  Table  3).  In  fact,  teachers  in  School  1 ranked 
second  lowest  with  respect  to  their  previous  experience  with  computers.  Analysis  of  qualitative  data  provides  an 
explanation  for  these  results.  The  TRTs  in  both  School  2 and  School  1 emphasize  their  role  in  helping  teachers  pass  the 
technology  competency  test,  while  the  other  TRTs  do  not  identify  this  role  as  their  major  responsibility  (see  Table  4). 
Teachers’  perception  of  the  TRTs  role  and  responsibilities  in  School  2 and  1 suggest  that  this  emphasis  by  the  TRT  in 
helping  teachers  pass  the  technology  competency  test  has  obviously  been  communicated  to  the  classroom  teachers, 
possibly  during  training  workshops  and/or  during  implementation  of  other  teaching  strategies. 

In  addition  to  providing  training  workshops,  technical  support,  and  troubleshooting,  the  majority  of  TRTs 
attempt  to  help  teachers  develop  lesson  plans  that  integrate  computers  into  instruction  using  word  processing, 
spreadsheets,  database  and  other  educational  software  (see  Table  5).  The  TRTs’  primary  instructional  strategy  for 
accomplishing  this  goal  is  to  model  the  teaching  of  the  various  computer  competencies  to  students  through  an  integrated 
lesson  within  different  subject  areas  (e.g.,  language  arts,  mathematics,  social  studies,  and  science). 

. . . .Well  anything  like  this  lesson  that  I have  created.  We  will  actually  do  this  lesson  at  our  workshop.  . . .A 
lot  of  what  I do  with  the  kid  I lead  the  teachers  through  first.  (TRT  School  2) 

....  When  they  come  to  the  lab  even  though  it  is  a generic  lesson,  it  is  oriented  to  some  subject.  It  is  database 
on  planets  or  a spreadsheet  on  nutrition,  so  I think  they  can  see  that  there  are  ways  to  do  this,  but  we  just  have 
to  make  it  specific  to  what  they  are  teaching  in  the  classroom. . . . (TRT  School  5) 

...  .If  I have  to  pull  them  in  there,  they  will  use  it.  You  know  once  they  see  how  it  can  be  done,  then  they 
will  be  more  willing  to  do  on  a more  regular  basis.  . . .Whenever  I come  up  with  something  I go  to  them.  . . . 
(TRT  School  1) 

The  integrated  lesson  in  most  cases  is  taught  by  the  TRT,  while  the  classroom  teacher  is  also  present  to  help  or  to 
observe.  In  a few  cases,  the  teachers  themselves  teach  the  integrated  lesson  while  the  TRT  is  present  to  help. 

All  TRTs  indicated  that  their  immediate  goal  with  respect  to  their  role  at  the  school  is  to  bring  technology  to 
teachers  and  students  and  to  make  them  feel  comfortable  with  technology. 

...  .1  think  my  role  is  to  prepare  students  and  teachers  for  computer  usage.  (TRT  School  2) 

. . . .Well,  have  a very  good  impact  on  teachers  use  of  computers.  (TRT  School  6) 

....  To  set  computer  lab  and  be  able  to  provide  technical  and  instructional  support.  (TRT  School  3) 

...  To  be  able  to  provide  technical  support  and  comfort  teachers  to  use  computer.  Also  help  teachers  and 
students  in  their  computer  skills  for  computer  competency  test.  (TRT  School  1) 

. . . Get  teachers  to  come  to  computer  lab  and  provide  technical  support.  (TRT  School  4) 

They  also  infer  that  their  long-term  goal  is  to  help  teachers  integrate  technology  into  their  classrooms  and  for  the 
respective  subject  teachers  to  assume  the  responsibilities  of  preparing  students  for  computer  usage.  None  of  TRTs  seem 
to  have  specific  short-term  or  long-term  performance  objectives  and/or  expectations  for  teachers  and/or  students  at  their 
respective  schools.  The  mandated  state  competency  exam  seems  to  have  driven  both  their  activities  and  workshops  at  the 
schools. 
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Q:  Do  you  have  specific  goal  or  objectives  for  this  year? 

My  aim  is  to  get  a support  system  in  place,  and  by  that  I mean  my  teachers  need  to  have  basic  skill  so  that 
when  we  bring  the  students  in  here,  two  of  us  know  the  skills  and  could  answer  the  questions.  . . I want  to  get 
as  many  of  them  passing  the  test  and  having  success  with  the  test.  . . .(TRT  School  2) 

I think  getting  with  the  teachers  in  their  planning  periods.  If  I can  sit  in  with  them,  seeing  what  they  are  doing, 
finding  out  their  problem  areas,  maybe  I can  enhance  their  problem  areas  of  how  they  teach  a lesson.  But  I think 
basically  working  with  them.  (TRT  School  3) 

We  started  off  with  the  operation  and  care  and  keyboarding  because  they  are  just  basic  information  about  the 
machines.  We  have  now  moved  into  more  complicated  item  types  such  as  database  and  spreadsheets.  In  our 
school  we  emphasize  on  computer  competencies.  . . . 

(TRT  School  6) 

As  indicated  above,  upon  establishing  the  computer  labs  at  their  schools,  all  six  TRTs  began  to  provide 
computer  training  workshops  for  teachers.  However,  interview  and  observation  data  show  that  none  of  the  TRTs 
attempted  to  conduct  a formal  needs  assessment  to  identify  the  areas  of  needs  for  training.  While  one  may  assume  that 
technology  coordinators  at  the  district  level  have  already  identified  the  needs  and  the  problem  areas  at  each  school,  the 
interview  data  with  the  coordinators  do  not  provide  any  evidence  for  this  assumption.  The  majority  of  TRTs,  however, 
did  indicate  that  they  already  knew  the  areas  of  needs  in  their  schools  due  to  their  familiarity  with  the  school  and  due  to 
their  close  relationship  with  the  teachers.  Two  of  the  TRTs  also  indicated  that  they  administered  a survey  at  the 
beginning  of  the  year  to  get  a sense  of  the  teachers'  perceptions  about  their  knowledge  and  skills  in  the  different  computer 
competency  areas. 

All  TRTs  also  established  an  hourly  computer  lab  schedule  for  their  schools.  This  schedule  is  used  to  encourage 
classroom  teachers  who  want  to  bring  their  classes  to  the  computer  lab  to  arrange  the  time  to  do  so.  The  TRTs  seem  to 
use  the  computer  lab  sign-up  sheet  to  identify  the  teachers  who  have  begun  to  think  about  integrating  computers  in  their 
curriculum.  A few  of  the  TRTs  also  keep  a sign-up  sheet  for  the  teachers  who  attend  the  training  workshops  in  order  to 
monitor  the  hours  of  training  workshops  that  each  teacher  completes.  However,  none  of  the  TRTs  seem  to  use  a formal 
and  continuous  monitoring  process  for  identifying  the  teachers’  progress;  nor  do  they  use  any  method  for  identifying  the 
effectiveness  of  the  strategies  that  they  have  employed  to  encourage  teachers  to  integrate  technology  into  their 
curriculum.  Again,  it  seems  that  the  number  of  teachers  who  are  able  to  pass  the  computer  competency  test  is  an 
implicit  indicator  of  success  for  both  the  TRTs  and  the  technology  coordinators  at  the  district  level.  Nevertheless,  all 
TRTs  seem  to  informally  monitor  the  teachers’  integration  of  computers  into  their  curriculum. 

The  Role  and  Responsibilities  of  Technology  Resource  Teacher:  What  are  They  Expected  to  Do  and  How  Do  They 
Perceive  their  Job? 

Tables  6a  and  6b  summarize  the  data  from  different  sources  regarding  the  roles  and  responsibilities  of  the  TRTs. 
As  Table  6a  indicates,  the  job  description  of  the  TRT  describes  the  instructional,  technical  and  administrative  dimensions 
of  the  job.  A content  analysis  of  the  job  description  also  reveals  that  the  instructional  aspect  of  the  position  is 
emphasized  more  than  the  other  two  aspects  of  technical  responsibilities  and  administrative  responsibilities  (8  statements 
on  instructional  responsibilities  as  opposed  to  4 statements  on  technical  responsibilities  and  5 statements  on 
administrative  responsibilities).  The  list  of  job  qualifications  also  requires  TRTs  to  have  a minimum  of  three  years 
teaching  experience.  Thereby,  indicating  the  pedagogical  importance  of  the  position. 

Table  6a  also  summarizes  the  perspectives  of  the  middle  school  principals  and  the  district  technology 
coordinators  about  the  roles  and  responsibilities  of  TRTs.  While  the  job  description  and  qualifications  for  TRTs 
emphasize  the  instructional  importance  of  the  position,  both  the  principals  and  the  technology  coordinators  indicated  the 
technical  and  interpersonal  skills  of  TRTs  to  be  the  major  factors  for  the  selection  of  the  individuals  for  this  position. 

The  majority  of  the  principals,  however,  point  out  that  they  preferred  to  select  one  of  their  technically  qualified  teachers 
at  the  school  for  the  position,  if  one  existed,  instead  of  hiring  someone  from  outside  of  the  school.  Their  reasons  for  this 
preference  include  the  easier  transition  process  and  the  familiarity  of  the  TRT  with  the  school’s  staff  and  its  context. 
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TRTs  were  also  asked  to  describe  their  perceptions  of  their  roles  and  responsibilities.  Table  6b  summarizes  the 
statements  that  are  used  by  each  TRT  to  describe  tlieir  job.  As  Table  6b  suggests,  while  there  are  some  differences  in  the 
perceptions  of  each  TRT  of  their  roles  and  responsibilities,  the  majority  of  TRTs  (83%)  agree  on  the  fact  that  they  spend 
70%  or  more  of  their  time  providing  technical  assistance  and  support  (e.g.,  setting  and  maintaining  the  lab, 
troubleshooting,  installing  software  and  hardware)  and  only  30%  or  less  of  their  time  providing  instructional  support. 

This  is  why  they  believe  that  they  should  have  spent  the  majority  of  their  time  providing  instructional  support.  Table  6 
also  lists  the  common  statements  that  teachers  at  different  schools  use  to  describe  their  perceptions  of  the  role  and 
responsibilities  of  the  TRT.  Although  there  are  some  differences  in  the  perception  of  teachers  in  different  schools  (see 
Table  4),  it  seems  that  teachers  generally  perceive  the  TRT  as  providing  more  instructional  support  than  technical 
support.  There  are  also  commonalities  between  the  TRTs*  perceptions  of  their  own  roles  and  responsibilities  and  the 
teachers’  perception  of  the  TRTs’  roles  and  responsibilities.  For  example,  the  TRT  in  School  3 perceives  self  as  a 
technical  support.  This  perception  is  consistent  with  the  perception  of  the  classroom  teachers  in  this  school  that  indicate 
technical  support  as  the  major  role  of  the  TRT.  The  TRT  in  School  2 perceives  the  TRT  role  as  helping  teachers  and 
students  to  pass  the  computer  competency  tests.  Teachers  in  this  school  also  perceive  the  major  role  of  the  TRT  to  be  in 
training  them  for  the  computer  competency  tests. 

The  Role  of  Technology  Resource  Teachers  in  Integration  of  Technology  into  the  Classroom 

How  do  teachers  use  technology?  The  results  from  the  general  survey  show  that  more  than  50%  of  teachers  who 
responded  to  our  survey  have  a home  computer,  60%  have  had  previous  computer  courses  or  workshops,  more  than  47% 
have  had  between  40  to  20  hours  of  computer  workshops  at  their  school,  and  81%  have  a computer  in  their  classroom. 

A chi-square  test  of  categorical  data  showed  no  significant  differences  between  schools  with  respect  to  the  above  variables 
(n-17). 


The  analysis  of  teachers*  responses  to  open-ended  questions  in  the  general  survey  reveals  some  useful 
information.  While  usage  of  computers  varies  slightly  from  one  school  to  the  next,  the  majority  of  the  teachers  who  are 
using  their  classroom  computers  indicate  that  they  are  mainly  using  word-processing  and  subject-matter  software  (see 
Table  7).  Enrichment/remediation  and  keyboarding  rank  next  as  the  third  and  the  fourth  most  common  applications  of 
the  computer  usage  in  the  classroom.  This  result  is  consistent  with  the  result  of  the  closed-ended  items  which  asks  a 
similar  question.  The  highest  percentage  of  classroom  computer  usage  is  for  remediation  and  enhancement  of  students 
learning.  Preparing  hand-outs  and  other  visual  presentation  rank  as  the  second  and  third  forms  of  classroom  computer 
usage  (see  Table  8).  The  interview  and  observation  data  also  reveal  that  word-processing,  games  and  subject-specific 
software  for  remediation  purposes  are  those  areas  of  computer  application  primarily  used  by  students  when  teachers  and 
students  are  using  in  computer  labs. 

Table  2 also  shows  that  teachers  across  different  schools  rate  their  word  processing  skills  as  the  strongest 
computer  skills,  while  they  rate  telecommunications  as  their  weakest  computer  skill  area.  (There  were  no  significant 
difference  among  the  six  schools  on  this  variable).  Comparison  of  this  result  with  the  above  outcome  attests  that 
teachers  are  using  computers  in  their  classrooms  in  the  areas  that  they  have  acquired  the  skills  and  knowledge  to  do  so. 

Analysis  of  interview  data  also  indicates  that  while  a number  of  teachers  at  each  school  responded  favorably  to 
the  TRTs*  initiatives  to  use  the  computer  lab  for  an  integrated  lesson,  the  majority  of  teachers  are  still  at  the  starting 
point  with  respect  to  both  computer  usage  and  the  integration  of  computers  into  their  curriculum. 

This  has  probably  been  harder  to  convince  people  that  it  is  possible.  I think  a lot  of  that  is  a function  of 
teachers'  skill  level.  I feel  like  I have  been  really  successful  in  some  situations  because  teachers  were  just  open 
and  wanting  (TRT,  School  6) 

A lot  of  teachers  are  coming  in  (lab).  . . . they  are  sending  interim  reports  on  the  computer.  I see  a lot  of  that 
type  of  things  going  on.  (TRT,  School  2) 
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Some  are  doing  it  now.  I have  got  some  that  are  just  wonderful.  They  do  a good  job  with  it  now.  Others  I 

think  it  is  just  a matter  of  they  need  to  be  shown  what  to  do  with  it.  They  are  so  used  to  doing  their  own 

things  that  it  would  be  hard  to  do  something  else.  (TRT,  School  1) 

There  are  some  folks  that  would  like  to  work  with  it  (computers)  but  still  hesitate  to  do  things  on  their  own. 

So  I have  to  go  and  pull  them  in.  Sometimes  you  just  have  to  go  and  say,  HI  think  we  can  do  this."  They  don’t 

know  what  they  can  do.  (TRT,  School  3) 

As  the  above  excerpts  show,  the  TRTs  think  it  will  take  time  for  teachers  to  learn  how  to  integrate  computers 
into  their  curriculum.  The  major  problems  that  TRTs  see  with  respect  to  teachers  who  are  reluctant  to  use  computers 
are:  not  enough  time  to  learn  about  computers,  not  knowing  the  application(s)  of  computers  in  their  subject  matter  area, 
and  not  seeing  the  benefit  of  computer  programs  for  students’  learning.  The  majority  of  TRTs  believe  that  all  of  the 
above  problems  can  be  solved  by  informally  approaching  teachers  and  helping  them  plan  an  integrated  lesson  and 
implement  it  successfully. 

Teachers  were  also  asked  to  explain  the  reasons  they  selected  certain  computer  skills  as  their  weakest  areas.  The 
majority  of  teachers  indicated  that  they  need  more  training,  time,  and  opportunity  to  practice  the  skills  that  they  have 
learned  in  order  to  be  able  to  use  computers  in  their  classroom  (see  Table  9).  This  is  their  perception,  although  at  the 
time  of  the  survey,  47%  of  teachers  had  already  completed  20  to  40  hours  of  computer  workshops  and  81%  had  taken 
previous  computer  courses.  From  these  figures  and  the  above  findings,  it  appears  that  training  workshops  are  only  the 
beginning  point  for  most  teachers.  In  order  for  teachers  to  use  computers  and  integrate  them  into  their  curriculum,  they 
need  practice  over  a long  period,  as  the  TRTs  pointed  out. 

How  do  teachers  feel  about  technology?  A total  of  206  teachers  across  schools  completed  the  CLA  survey  at 
the  end  of  the  1994-95  school  year.  This  was  after  having  TRTs  in  their  respective  schools  during  that  year.  The  survey 
contained  items  related  to  liking  computers,  valuing  computers  for  society  and  education;  anxiety  about  using  computers; 
confidence  with  regard  to  learning  and  using  computers;  and  perceptions  of  gender  appropriateness  of  computers. 

Teachers  were  asked  to  rate  the  items  from  “Strongly  Agree”  to  “Strongly  Disagree”.  In  addition  to  Likert-scale  items, 
teachers  were  also  asked  several  background  questions.  A summary  of  the  responses  to  these  questions  is  presented  in 
Table  10. 

The  results  of  attitude  survey  show  that  teachers  have  a fairly  positive  attitude  toward  computers.  Appendix  B 
summarizes  the  means  of  all  teachers  responses  to  Computer  Attitude  Survey.  As  the  means  scores  shows  teachers  seem 
to  like  computers,  value  computers  for  education  and  society  and  have  confidence  about  learning  and  using  computers. 
However,  they  still  have  some  anxiety  about  computers  (Average  M=  4.00  where  5=Strongly  Disagree).  A repeated 
measure  of  attitude  is  required  to  determine  any  changes  in  teachers’  attitude  over  time. 

A correlation  analysis  of  different  items  related  liking  computers,  valuing  computers  for  society  and 
education,  anxiety  about  using  computers,  confidence  with  regard  to  learning  and  using  computers,  previous  experience 
with  computers,  ratings  of  computer  skills  at  present  and  before  the  school  year,  and  current  usage  of  computers  in  the 
classroom  revealed  some  interesting  findings.  Tables  1 1 to  16  summarize  the  results  of  this  analysis.  As  these  tables 
show,  teachers  who  rated  their  computer  skills  higher,  liked  computers  more,  had  more  confidence  for  learning 
computers,  valued  computers  for  the  society  and  education  more  than  others,  and  had  less  anxiety  about  using  computers. 
Findings  also  showed  correlations  between  confidence  for  learning  computers  and  the  level  of  anxiety  for  using 
computers.  Teachers  who  felt  more  confident  learning  and  using  computers  tended  to  have  less  anxiety  for  computers. 
Teachers  who  had  previous  experience  with  operating  computers  also  showed  more  confidence  for  using  computers.  Table 
16  shows  that  teachers  who  had  completed  more  hours  of  training  workshops  tended  to  rate  their  computer  skills  higher 
than  others. 

An  analysis  of  multivariance  was  also  conducted  to  investigate  the  combined  effects  of  different  categories  of 
attitude  toward  computers  at  different  schools,  the  usage  of  computers  in  the  classroom,  different  levels  of  previous 
computer  experiences,  and  different  levels  of  computer  skills.  Since  the  response  rate  was  not  equal  for  each  school,  a 
random  sample  of  24  responses  was  selected  from  each  school  for  this  analysis.  Tables  17  through  21  show  the  means 
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and  standard  deviation  for  each  dependent  variable.  The  result  of  the  multivariate  tests  are  also  presented  in  Table  22.  As 
Table  17  shows,  the  difference  between  schools  across  variables  such  as  liking  computers,  value  for  computers  in 
education,  confidence  for  learning  and  using  computers,  and  anxiety  for  computers  is  not  significant. 

We  also  investigated  the  combined  effects  of  the  above  variables  represented  in  the  attitude  survey  on  the  current 
usage  of  computers  in  the  classroom.  Table  18  shows  means  and  standard  deviations  for  each  dependent  variable.  The 
results  of  a multivariate  test  is  also  presented  in  Table  22.  As  the  table  shows,  the  difference  between  the  teachers  who 
are  currently  using  computers  and  the  teachers  who  are  using  computers  across  different  ??  attitude  measures  was 
significant.  The  univariate  test  shows  that  teachers  who  are  currently  using  computers  in  their  classroom,  like 
computers  more,  and  value  computers  for  education  more  than  those  who  are  not  using  computers  in  their  classroom. 

The  anxiety  and  the  confidence  for  using  computers  is  not,  however,  significantly  different  for  those  who  are  using 
computers  in  their  classrooms  and  those  who  are  not  using  computers  in  their  classrooms. 

Table  22  also  shows  that  there  is  a significant  difference  between  teachers  who  had  previous  experiences  with 
operating  computers  and  those  who  did  not  on  categories  such  as  liking  computers,  confidence  for  using  computers,  and 
anxiety  for  computers.  In  other  words,  teachers  who  had  over  a year  of  previous  experience  with  computers  liked 
computers  significantly  more,  had  less  anxiety  toward  computers,  and  had  a higher  level  of  confidence  for  using 
computers  than  those  who  had  no  previous  experience  (see  Table  19  ).  However,  this  difference  was  not  significant  for 
categories  such  as  value  for  computer  in  education  and  confidence  for  learning  computers. 

Finally,  as  presented  in  Table  22,  there  was  a significant  difference  between  the  attitude  of  teachers  toward 
computers  (on  all  categories  except  for  the  value  for  computer  in  education)  and  the  level  of  computer  skills.  In  other 
words,  teachers  who  rated  their  computer  skills  highest  before  and  after  the  school  year  liked  computers  more,  had  more 
confidence  learning  and  using  computers,  and  felt  less  anxiety  towards  computers  (see  Table  20  and  21). 

In  sum,  although  the  statistical  analysis  of  the  attitude  survey  revealed  some  very  interesting  results,  it  failed  to 
demonstrate  any  significant  differences  between  attitude  of  teachers  toward  computers  at  different  schools.  Usage  of 
computers  in  the  classroom,  previous  computer  experience,  and  skills  in  using  computers  are  identified  as  factors  that  had 
effects  on  teachers'  general  attitudes  toward  computers. 


Discussion 

This  study  has  been  conducted  to  explore  the  roles  and  responsibilities  of  the  TRT,  and  the  effects  that  the 
TRT’s  role  has  on  the  integration  of  technology  in  the  classroom.  It  also  sought  to  examine  how  the  TRTs  role  differed 
from  that  expected  of  instructional  designers/technologists,  if  they  were  to  be  involved  in  public  schools;  and  to  explore 
what  an  instructional  technologist  would  have  done  if  he/she  was  assigned  to  this  position. 

The  study  showed  that  the  role  and  responsibilities  of  TRTs  are  primarily  instructional,  although  technical  and 
administrative  responsibilities  are  also  expected.  Excellent  communication  skills,  the  ability  to  understand  the  dynamics 
of  teaching  and  the  role  of  teachers,  and  being  able  to  work  closely  with  the  classroom  teachers  appeared  to  be  vital 
characteristics  contributing  to  the  success  of  the  TRT,  particularly  as  they  affected  the  classroom  teachers'  acceptance  of 
and  openness  towards  the  technology.  In  addition,  familiarity  with  the  school  context  appears  to  enhance  the  readiness 
for  success,  both  through  the  TRT’s  ability  to  understand  and  analyze  where  the  teachers’  progress  lies,  and  through  the 
classroom  teachers'  comfort  level  for  working  with  the  TRT.  Although  the  TRTs  estimated  that  they  spent  around  75% 
of  their  time  on  technical  support  and  the  county  administrators  expected  a high  level  of  technical  expertise  in  the 
requirements  for  selecting  a TRT,  the  study  indicated  that  the  nature  of  the  TRTs  role  was  more  instructional  than 
technical.  For  example,  the  teachers’  expectations  of  the  TRT  was  one  of  instructional  support  through  training, 
workshops,  and  demonstration  of  the  applications  of  the  computer.  Furthermore,  the  TRTs  felt  that  their  next  phase 
would  result  in  much  less  technical  work  and  much  more  instructional  work  in  the  integration  of  technology  into  the 
classroom. 

As  an  instructional  support  system,  each  TRT  used  some  strategics  to  help  teachers  in  his  or  her  school 
integrate  computers  into  their  curriculum.  Although  TRTs'  instructional  strategics  were  somewhat  different,  their  primary 
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instructional  strategics  were:  providing  training  workshops,  teaching  integrated  lesson  to  students  while  teachers  were 
present,  approaching  teachers  informally  to  help  them  and  give  them  new  ideas,  and  encouraging  teachers  to  schedule 
their  classes  for  the  computer  lab.  However,  while  teaching  computer  skills  to  teachers  and  students  was  the  TRTs’ 
major  instructional  activity,  the  findings  of  the  study  indicated  that  TRTs  did  not  conduct  a needs  analysis  at  the  school. 
They  did  not  prioritize  the  needs,  they  did  not  have  specific  objectives,  nor  did  they  have  a planned  action  for 
implementation  and  evaluation.  Instead,  they  applied  an  intuitive  and  informal  understanding  of  the  needs  to  initiate  the 
strategies  or  support.  They  provided  training  workshops  based  on  their  own  understanding  of  the  teachers’  needs, 
informally  monitored  the  process,  and  used  more  of  the  collegial  relationships  with  teachers  to  help  them  integrate 
technology. 

As  an  instructional  designer,  when  one  approaches  the  task,  one  approaches  it  first  by  identifying  the  problem 
through  a formal  analysis  of  needs  to  determine  if  there  is  an  instructional  solution  to  the  problem.  Instructional 
designers  then  proceed  with  the  design  of  instruction  to  meet  the  needs  of  the  learners,  monitor  the  process,  evaluate  the 
results,  and  revise,  if  necessary.  While  it  is  premature  to  determine  if  TRTs’  strategies  for  helping  teachers  integrate 
technology  was  effective,  data  show  that  the  TRTs  are  able  to  develop  a rapport  with  teachers  which  enabled  teachers  to 
feel  comfortable  working  with  the  TRT,  working  in  the  computer  lab,  and  to  more  willingly  approach  utilizing 
technology  in  their  instruction. 

Had  the  TRTs  been  trained  in  the  field  of  instructional  design,  they  would  have  combined  their  knowledge  of  the 
school  environment,  their  teaching  experience  in  the  school,  and  familiarity  with  the  teachers  with  an  analytical, 
systematic,  and  evaluative  approach  in  their  efforts  to  help  teachers  both:  (1)  integrate  technology,  and  (2)  learn  the 
required  technology  skills  necessary  to  do  so.  Since  the  goals  of  the  TRTs  were  derived  from  the  state's  predefined  and 
mandated  competency  test,  the  TRTs’  approach  was  reactive  rather  than  proactive.  A more  proactive,  systemic,  as  well 
as  systematic  approach  could  have  resulted  in  more  integration  of  computers  into  curriculum.  The  lack  of  a formal 
statement  of  needs  that  is  based  on  the  analysis  of  the  problem  in  each  school  and  formal  measures  did  not  allow  TRTs  to 
monitor  or  assess  the  achievement  or  appropriateness  of  their  goals  and  strategies,  nor  did  it  provide  them  with 
opportunity  to  revise  their  plan  of  action  . 


Implications 

If  public  schools  would  like  to  use  the  time  and  the  money  efficiently,  they  need  to  use  professionals  who  have 
not  only  the  characteristics  of  the  TRTs  but  also  skills  in  instructional  design.  The  study  has  also  implications  for 
instructional  technology  field. 

First  of  all,  in  the  field  of  instructional  design/technology,  more  emphasis  needs  to  be  placed  on  the  role  of 
instructional  designer  in  the  public  schools.  The  results  of  this  study  suggest  that  public  schools  may  be  ready  to  use 
instructional  design  skills  that  for  a long  period  of  time  had  not  been  utilized.  From  the  findings  of  this  study,  the 
following  suggestions  may  be  pertinent  to  the  field. 

1 . Recruit  F.riucators  to  Become  Potential  Instructional  Designers  in  the  Public  Schools 

In  order  for  instructional  design  to  be  successfully  implemented  in  schools,  the  field  should  place  more  emphasis 
on  public  school  settings  and  recruit  newcomers  to  the  field  directly  from  the  pool  of  educators  from  the  school  system. 
The  answer  may  not  lie  in  simply  sending  newly  trained  instructional  designers  into  the  schools,  particularly  since  this 
study  indicates  the  importance  being  familiar  with  the  school  system,  the  school  context,  and  the  characteristics  of 
teachers  and  teaching  practices. 

2.  Include  a Public  School  Internship  Component  to  Instructional  Technology  Programs 

Another  way  that  the  instructional  design  field  can  aim  their  efforts  towards  helping  public  schools  is  to  include 
a public  school  internship  requirement  in  current  programs.  The  majority  of  graduate  programs  in  instructional 
design/technology  usually  require  students  to  complete  an  internship  in  business,  industry,  government,  or  military 
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settings.  Potential  applicants  interested  in  public  school  settings  should  be  required  to  complete  their  internship  in  a 
public  school  or  schools. 

3.  Provide  Inservice  Instructional  Technology  Training  for  Public  School  Teachers 

Given  that  there  is  an  overwhelming  demand  for  public  school  educators  and  their  students  to  have  computer 
skills,  the  instructional  technology  field  can  take  the  leadership  and  responsibility  for  providing  that  training.  One  of  the 
problems  that  has  kept  instructional  designers  from  public  education  has  been  the  lack  of  funding  to  pay  for  such 
expertise.  Schools  and  the  instructional  design  field  should  become  partners  in  their  attempts  to  obtain  grants  and 
funding  means  to  support  such  training  efforts. 

4 Target  Potential  Employment  Opportunities  Such  as  Technology  Coordinator.  Technology  Resource  Teacher,  and 
Other  Public  School  Technology  Positions 


The  results  of  this  study  combined  with  current  trends  in  education  .suggest  that  the  public  school  community  is 
ready  to  use  professionals  from  the  field  of  instructional  design.  While  the  schools  in  this  study  have  such  positions  in 
place,  other  schools  in  the  state  and  across  the  nation  will  be  working  to  solve  technology  integration  problems  that  will 
probably  result  in  similar  job  positions.  The  field  of  instructional  design  may  have  a rich  opportunity  at  last  to  affect 
public  schools. 
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Appendix  A (continued) 
Computer  Survey 


Instructions:  Mark  all  responses  on  your  bubble  sheet  using  a #2  pencil  only.  The  bubble  sheets  are  anonymous 
and  do  not  require  your  name,  birthdate,  identification  number  or  grade  or  education.  However,  we  would  appreciate  it  if 
you  would  please  indicate  whether  you  are  male  or  female  in  the  appropriate  area  on  the  bubble  sheet.  Please  do  not 
make  marks  on  this  questionnaire. 

Answer  each  question  as  it  applies  to  you  at  this  point  in  your  learning  and  teaching  experiences.  Your  answers  will  not 
be  shared  with  anyone  and  will  be  kept  anonymous. 


1 . What  grade  levels  do  you  teach? 

a.  6th  grade 

b.  7th  grade 

c.  8th  grade 

2.  What  subject  areas  do  you  teach? 

a.  Mathematics 

b.  Social  Studies 

c.  Science 

d.  Language  Arts 

e.  Other 

3.  Are  you  seeking  CEU  credit  for  the  computer  competency  workshops  that  you  take? 

a.  Yes 

b.  No 

4.  How  many  hours  of  computer  workshop  training  have  you  completed? 

a.  Less  than  10  hours 

b.  10-20  hours 

c.  20-30  hours 

d.  30-40  hours 

5.  Are  you  currently  using  computers  in  your  classroom? 

a.  Yes 

b.  No 

6.  Before  this  year,  what  previous  experience  have  you  had  in  operating  computers? 

a.  0 

b.  Few  days  to  1 month 

c.  1-6  months 

d.  7 months  to  a year 

e.  Over  a year 
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7. 


Describe  your  home  computer: 


a.  Macintosh 

b.  IBM  Compatible 

c.  Apple  II 

d.  Other 

e.  None 

8.  How  would  you  rate  your  computer  skills  before  the  school  year  started? 

a.  Nonexistent 

b.  Poor 

c.  Average 

d.  Good 

e.  Excellent 

9.  How  would  you  rate  your  ability  to  use  a computer  now? 

a.  Nonexistent 

b.  Poor 

c.  Average 

d.  Good 

e.  Excellent 


Answer  the  next  set  of  items  according  to  how  much  you  agree  or  disagree  with  the  statements  as  follows: 

a.  strongly  agree 

b.  agree 

c.  neutral 

d.  disagree 

e.  strongly  disagree 

10.  Knowing  how  to  use  computers  is  a worthwhile  and  necessary  skill. 

11.  I like  using  computers. 

12.  I feel  confident  about  my  ability  to  learn  about  computers. 

13.  Working  with  a computer  makes  me  nervous. 

14.  I will  use  my  knowledge  of  computers  in  many  ways  as  a teacher. 

15.  Using  a computer  is  more  important  for  males  than  females. 

16.  I like  using  computers  in  my  school  work. 

17.  I wish  I could  use  computers  more  frequently  at  the  school. 
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18.  I get  a sinking  feeling  when  I think  of  trying  to  use  a computer. 

a.  strongly  agree 

b.  agree 

c.  neutral 

d.  disagree 

e.  strongly  disagree 

19.  Once  I start  to  work  with  the  computer,  I would  find  it  hard  to  stop. 

20.  Computers  make  me  feel  stupid. 

21 . If  a problem  is  left  unsolved  in  a computer  workshop  or  in  class,  I would  continue  to  think  about  it  afterwards. 

22.  More  men  than  women  have  the  ability  to  become  computer  scientists. 

23.  Teaching  using  computers  would  be  very  interesting. 

24.  I don't  expect  to  use  computers  in  my  classroom. 

25.  I look  forward  to  using  the  computers  at  school. 

26.  I'm  not  the  type  to  do  well  with  computers. 

27.  I feel  comfortable  using  computers. 

28.  Working  with  computers  is  boring. 

29.  Using  computers  is  more  enjoyable  for  males  than  females. 

30.  When  there  is  a problem  with  a computer  program  I can’t  immediately  solve,  I would  stick  with  it  until  I have 

the  answer. 

3 1 . Learning  about  computers  is  a worthwhile  and  necessary  subject  for  all  teachers. 

32.  Computers  make  me  feel  uncomfortable. 

33.  It  is  important  to  know  how  to  use  computers  in  order  to  get  any  teaching  position. 

34.  I know  that  if  I work  hard  to  learn  about  computers,  I can  do  well. 

35.  Females  can  do  as  well  as  males  in  learning  about  computers. 

36.  Computers  make  me  feel  uneasy  and  confused. 

37.  I think  working  with  computers  would  be  both  enjoyable  and  stimulating. 

38.  I think  using  a computer  would  be  difficult  for  me. 

39.  Working  with  computers  is  more  for  males  than  females. 


rr 


? * o 


531 


40.  I am  able  to  do  as  well  working  with  computers  as  most  of  my  fellow  teachers. 

a.  strongly  agree 

b.  agree 

c.  neutral 

d.  disagree 

e.  strongly  disagree 

41.  I will  probably  need  to  know  how  to  use  a computer  in  my  classroom. 

42.  Computers  are  gaining  too  much  control  over  people’s  lives. 

43.  Supplying  every  student  with  a computer  is  a worthy  educational  objective. 

44.  Teachers  should  demand  that  they  be  taught  how  to  use  a computer  in  their  classrooms. 

45.  Computers  will  require  learners  to  become  active  in  their  learning. 

46.  Computer  instruction  will  deny  students  the  opportunity  to  reason  with  others. 

47.  Using  computers  as  a teaching  tool  puts  too  much  additional  work  on  already  overburdened  teachers. 

48.  If  we  do  not  use  computers  in  school  instruction,  our  students  will  grow  up  illiterate  and  deprived  of  a basic 
skill. 

49.  If  my  school  district  had  the  money,  I would  insist  that  they  buy  computers  in  most  every  school  subject. 

50.  Computers  will  increase  the  amount  of  stress  and  anxiety  teachers  experience  in  schools. 

5 1 . Computers  will  decrease  the  amount  of  teacher-pupil  interaction  in  schools. 

52.  Computers  will  isolate  students  from  one  another. 

53.  I object  to  all  the  attention  being  given  to  computer  technology  because  it  detracts  from  the  real  problems  now 
faced  by  teachers. 

54.  Computers  can  improve  learning  of  higher-order  skills. 

55.  Computers  will  displace  teachers. 

56.  Computers  will  dehumanize  teaching. 

57.  Our  country  would  be  better  off  if  there  were  no  computers. 

58.  Someday  I will  have  a computer  in  my  home. 

59.  Computers  wil1  improve  education. 
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Appendix  B 

Teachers*  Attitude  Toward  Computers-Mean  and  Standard  Deviation  Scores. 


Likine  Computers 

M 

_SD 

11. 

I like  using  computers. 

1.63 

.94 

16. 

I like  using  computers  in  my  school  work. 

1.82 

1.02 

17. 

I wish  I couid  use  computers  more  frequently  at  the  school. 

1.82 

.94 

19. 

Once  I start  to  work  with  the  computer,  I would  find  it  hard  to  stop. 

2.40 

1.14 

21. 

If  a problem  is  left  unsolved  in  a computer  workshop  or  in  class, 
I would  continue  to  think  about  it  afterwards. 

2.41 

1.10 

23. 

Teaching  using  computers  would  be  very  interesting. 

2.09 

1.07 

25. 

I look  forward  to  using  the  computers  at  school. 

1.90 

.97 

30. 

When  there  is  a problem  with  a computer  program  I can't 
immediately  solve,  I would  stick  with  it  until  I have  the  answer. 

2.53 

1.11 

37. 

I think  working  with  computers  would  be  both  enjoyable  and 
stimulating. 

1.88 

.91 

58. 

Someday  I will  have  a computer  in  my  home. 

1.63 

.95 

28. 

Working  with  computers  is  boring.  (Negative  item) 

4.25 

.91 

Value  of  Computer  for  Education 



SD 

Positive  Items 

14. 

I will  use  my  knowledge  of  computers  in  many  ways  as  a teacher. 

1.88 

1.03 

31. 

Learning  about  computers  is  a worthwhile  and  necessary  subject 
for  all  teachers. 

1.58 

.90 

33. 

It  is  important  to  know  how  to  use  computers  in  order  to  get  any 
teaching  position. 

2.80 

1.16 

43. 

Supplying  every  student  with  a microcomputer  is  a worthy 
educational  objective. 

2.12 

1.12 

44. 

Teachers  should  demand  that  they  be  taught  how  to  use 
microcomputers  in  their  classrooms. 

2.09 

.95 

45. 

Microcomputers  will  require  learners  to  become  active 
in  their  learning. 

2.02 

.96 

48. 

If  we  do  not  use  microcomputers  in  school  instruction, 

our  students  will  grow  up  illiterate  and  deprived  of  a basic  skill. 

2.60 

1.23 

49. 

If  my  school  district  had  the  money,  I would  insist  that  they 
buy  microcomputers  in  most  every  school  subject. 

2.18 

1.07 

54. 

Computers  can  improve  learning  of  higher-order  skills. 

1.95 

.89 

57. 

Our  country  would  be  better  off  if  there  were  no  computers. 

1.95 

.89 

59. 

Computers  will  improve  education. 

1.70 

.90 

Negative  Items 

46. 

Microcomputer  instruction  will  deny  students  the  opportunity 
to  reason  with  others. 

3.75 

1.03 

47. 

Using  microcomputers  as  a teaching  tool  puts  too  much  additional 
work  on  already  overburdened  teachers. 

3.56 

1.16 

50. 

Microcomputers  will  increase  the  amount  of  stress  and  anxiety 
teachers  experience  in  schools. 

3.51 

1.10 

51. 

Microcomputers  will  decrease  the  amount  of  teacher-pupil 
interaction  in  schools. 

3.71 

1.05 

52. 

Microcomputers  will  isolate  students  from  one  another. 

3.82 

.91 

53. 

I object  to  all  the  attention  being  given  to  computer  technology 
because  it  detracts  from  the  real  problems  now  faced  by  teachers. 

3.60 

1.17 
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55. 

Computers  will  displace  teachers. 

4.14 

.96 

56. 

Computers  will  dehumanize  teaching. 

3.96 

1.00 

Value  of  Computer  for  Society 

M 

3D 

Positive  Items 

10. 

Knowing  how  to  use  computers  is  a worthwhile  and  necessary 
skill. 

1.29 

.71 

41. 

I will  probably  need  to  know  how  to  use  a computer  in  my 
classroom. 

1.63 

.93 

Negative  Items 

24. 

I don’t  expect  to  use  computers  in  my  classroom.  (Negative) 

4.37 

.91 

42. 

Computers  are  gaining  too  much  control  over  people's 
lives.  (Negative) 

3.41 

1.20 

Confidence  about  Learning  Computers 

M 

SD 

12. 

I feel  confident  about  my  ability  to  learn  about  computers. 

1.78 

.96 

34. 

I know  that  if  I work  hard  to  learn  about  computers, 
I can  do  well. 

1.60 

.71 

Confidence  About  Using  Computers 

M 

SD 

27. 

I feel  comfortable  using  computers. 

2.11 

1.14 

40. 

I am  able  to  do  as  well  working  with  computers  as  most  of 
my  fellow  teachers. 

1.98 

1.02 

Anxiety  (or  Lack  of  it)  about  Computers 

M 

_$P 

Negative  Items 

13. 

Working  with  a computer  would  make  me  nervous. 

3.52 

1.31 

18. 

I get  a sinking  feeling  when  I think  of  trying  to  use  a computer. 

4.06 

1.15 

20. 

Computers  make  me  feel  stupid. 

4.04 

1.15 

26. 

I'm  not  the  type  to  do  well  with  computers. 

4.13 

1.00 

32. 

Computers  make  me  feel  uncomfortable. 

3.93 

1.16 

36. 

Computers  make  me  feel  uneasy  and  confused. 

3.97 

1.11 

38. 

I think  using  a computer  would  be  difficult  for  me. 

4.09 

1.04 

Perception  about  Gender-Appropriate  of  Computer  Use 

M 

SD 

15. 

Using  a computer  is  more  important  for  males  than  females. 

4.58 

.92 

22. 

More  men  than  women  have  the  ability  to  become  computer 
scientists. 

4.61 

.73 

29. 

Using  computers  is  more  enjoyable  for  males  than  females. 

4.54 

.83 

35. 

Females  can  do  as  well  as  males  in  learning  about  computers. 

1.47 

.93 

39. 

Working  with  computers  is  more  for  males  than  females. 

4.60 

.72 

Note:  For  positively-worded  statments  l=strongly  agree  and  for  negatively-worded  statements  5=strongly  disagree. 
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Table  1 

Cross  tabulation  of  computer  competency  exams  taken  by  teachers  by  school. 


Schools 


Question 

i 

2 

3 

4 

5 

6 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

How  many  hours  of 

16.0 

17.0 

16.0 

17.0 

17.0 

17.0 

computer  workshops 

have  completed? 

Which  of  the  following 

computer  competency 

exams  have  you  taken? 

• Keyboarding 

14.3 

25.7 

8.6 

11.4 

11.4 

28.6 

• Operating 

19.2 

34.6 

4.5 

13.6 

9.1 

13.6 

•Word- 

22.7 

36.4 

4.5 

13.6 

9.1 

13.6 

processing 

• Spreadsheet 

23.1 

30.8 

3.8 

7.7 

11.5 

23.1 

• Data  base 

23.8 

42.9 

0.0 

4.8 

9.0 

19.0 

Table  2 


Means  and  standard  deviations  of  teachers1  different  computer  skills  by  school. 

Schools 


Question 

i 

(n=17) 

M 

(SD) 

2 

(n=17) 

M 

(SD) 

3 

(n=17) 

M 

(SD) 

4 

(n=17) 

M 

(SD) 

5 

(n=17) 

M 

(SD) 

6 

(n—1 7) 
M 

(SD) 

List  the  following  computers  skills 
in  order  of  your  strongest  areas  to 
your  weakest  area  (1=  strongest  and 
9=  weakest) 

• Terms  and  operations 

2.7 

4.0 

3.0 

3.4 

3.8 

4.6 

(1.9) 

(2.3) 

(2.0) 

(1.6) 

(2.3) 

(2.6) 

• Keyboarding 

3.6 

2.2 

2.3 

1.7 

4.3 

2.7 

(2.7) 

(2.0) 

(2.1) 

(1-2) 

(2.6) 

(2.5) 

• Societal  uses 

5.8 

5.5 

5.5 

6.4 

6.4 

4.3 

(1.9) 

(2.5) 

(2.6) 

(1.7) 

(2.0) 

(2.1) 

• Ethics 

5.6 

5.5 

5.2 

5.7 

6.4 

3.6 

(2.4) 

(2.7) 

(2.3) 

(2.5) 

(2.5) 

(2.3) 

• Word-processing 

3.8 

2.7 

1.7 

2.0 

3.1 

3-6 

(2.4) 

(2.2) 

(1.0) 

(0.9) 

(2.5) 

(2.3) 

• Data  bases 

6.0 

5.2 

4.5 

6.0 

5.6 

6.0 

(2.0) 

(1.8) 

(1.8) 

(19) 

(2.2) 

(1.6) 

• Spreadsheets 

5.6 

5.2 

4.6 

5.0 

5.2 

6.1 

(2.1) 

(1.8) 

(2.7) 

(2.3) 

(2.4) 

(1.6) 

• Curriculum  software 

5.6 

5.7 

5.0 

5.7 

4.6 

5.1 

use 

(2.2) 

(2.2) 

(2.3) 

(1.8) 

(1-8) 

(2.5) 

• Telecomputing 

8.0 

8.2 

7.7 

8.2 

6.8 

8.3 

(2.7) 

(0.8) 

(2.4) 

(1.3) 

(2.6) 

(1.2) 
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Table  3 

Cross  tabulation  of  previous  experiences  in  computer  by  school. 


Schools 


Questions 

1 

2 

3 

4 

5 

6 

(n=I7) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

Have  you  had  any  previous 
computer  course/workshop  prior  to 
the  school-based  computer  training 
program? 

14.1 

20.3 

17.2 

12.5 

15-6 

20.3 

Describe  your  computer  skills  prior 
to  your  training  this  school  year. 

17.0 

16.0 

17.0 

17.0 

17.0 

17.0 

17.0  %•  I feel  I had  all 

58.0  %•  I feel  I had  some 

(4) 

(1) 

(3) 

(3) 

(6) 

(0) 

25.0  %•  I feel  I had  none 

(10) 

(8) 

(10) 

(9) 

(6) 

(15) 

(3) 

(7) 

HI 

(5) 

(5) 

Hi 

Please  describe  your  computer  skills 

17.0 

17.0 

15.0 

17.0 

17.0 

17.0 

at  the  present  time. 

• 

27.0%*  Confidence  in  all 

(4) 

(6) 

(4) 

(3) 

(7) 

(3) 

49.0%*  Confidence  in  some 

(8) 

(8) 

(9) 

(11) 

(3) 

(10) 

24.0%*  Need  additional 

(5) 

(3) 

(2) 

_£> 

HI 

i4) 

Table  4 

Summary  of  Teachers'  Perception  of  the  Role  and  Responsibilities  of  Resource  Teachers  and  Summary  of  Resource 
Teacher’s  Perception  of  his/her  Role  and  Responsibilities. 


School 

Teachers1  Perception  of 
Resource  Teacher 

% 

Resource  Teacher's  Perception 
of  his/her  Role 

School 

• Always  there  to  help  and 

52 

• Teach  teachers  to  plan  lessons  for 

1 

suggest/ Advise/Assist 

teaching  computer  competency  test 

teachers/students 

• Help  teachers  to  come  up  with  new 

• Conducts  workshops/Provides 

16.6 

ideas  to  use  technology  in  their 

training  sessions 

curriculum 

• Staff  developer 

13.0 

• Help  teachers  in  computer  lab 

• Teaches  specific  lessons 

8.7 

• Maintain  computer  lab  operation 

• Direct  group 

8.7 

• Troubleshooting 

instruction/Coordinates 

activities/Newsletters 

• Problem-fixer/Trouble  shooter 

8.7 

with  computers 

• Mentor/Facilitator 

4.3 

• Answers  questions  concerning 

4.3 

computer  programs 

• Willing  to  help  in  any  area 

4.3 

tv  r 
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Table  4.  (continued) 


School 

Teachers1  Perception  of 
Resource  Teacher 

% 

Resource  Teacher’s  Perception 
of  his/her  Role 

School 

• Assist  or  aids  teachers  in  the  use 

30.8 

• Train  teacher  for  computer 

2 

and  teaching  of  computers/ 

competency  test 

teaches  computing  skills 

• Prepare  students  for  computer 

• Creates  certificates/helpful  for 

7.7 

competency  tests 

competency  exam 

• Install  New  Software 

• Helps/Assists  with  technology 

7.7 

• Troubleshoot 

lessons 

• Keep  track  of  hardware  and  software 

• Facilitate  instruct ion/Technology 

7.7 

facilitator 

• Aids  in  teacher  performance 

3.8 

• Provides  workshops 

3.8 

• Keeps  hardware  running,  installs 

3.8 

and  orders  software 

• Manages  computer  lab 

3.8 

School 

• Troubleshooter/Fixer 

27.8 

• Provide  technical  support 

3 

• Helps  teacher/students  with 

22.2 

• Train  students  for  taking  computer 

computer  skills  and  competency 

competency  test 

test 

•Facilitator 

11.1 

• Tutor/Trainer 

11.1 

• Provides  support  to 

11.1 

teachers/Answers  questions 

• Gives  lessons  to  kids  and  teachers 

11.1 

• The  one  and  only  genius 

• Runs  computer  lab 

5.6 

• Seems  quite  knowledgeable  in 

5.6 

most  aspects 

5.6 

• Discusses  and  teaches  NC 

computer  competency  objectives 

5.6 

r 
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Table  4 (continued) 


School 

Teachers*  Perception  of 

% 

Resource  Teacher’s  Perception 

Resource  Teacher 

of  her/his  Role 

School 

• Teaches  computer  skills  to  both 

32.3 

• Help  teachers  plan  lessons  to 

4 

staff  and  students/Helps  teacher 

integrate  computer 

and  students  become  computer 

• Help  and  teach  computer  skills  to 

literate 

students  for  computer  competency 

• Helper/Never  too  busy  to 

29.0 

test 

help/Most  helpful  to  all/God's 

• Maintain  computer  lab  operation 

Send  to  us/Invaluable/Essential 

• Train  teachers  for  taking  computer 

• Technical  advisor/trouble  shooter 

12.9 

competency  test 

• Computer  specialist/Responds  to 

• Provide  workshop 

all  of  our  computer  needs 

. 9.7 

• Resource  for  software/Helps  us 

find  software 

9.7 

• Life  saver/Our  savior 

• Offers  suggestions/provides 

6.5 

support 

6.5 

• Facilitator 

• Helpful  with  training 

6.5 

• Assists  teachers  in  planning  for 

6.5 

computer  lab 

3.2 

• Guides  classroom  instruction 

3.2 

School 

• Assist  with  student 

18.8 

• Basically  troubleshooting  in 

5 

instruction/Teaches  computer 

classrooms 

language  to  students/Design 

• I want  to  be  a friendly  person  that 

lesson  for  students 

teachers  feel  comfortable  coming  to 

• Facilitator  for  computers  and 

18.8 

• I don  t want  to  be  perceived  as  a nuts 

computer  instruction 

and  bolts  hardware  person 

• Work  with  teachers  and  students 

14.6 

on  computer  skills/usage 

• Great  resource  to  help  and  support 

12.5 

• Trainer/Provide  training  for 

teachers/  Provide  workshop 

10.4 

• Troubleshooter/Technician 

• An  expert  to  consult  with 

8.3 

• Help  students  and  teachers  with 

8.3 

any  questions/new  information 

8.3 

• Assist  teachers  in  finding 



materials  for  use  with  classes 

6.3 

r r '■) 
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Table  4 (continued) 


School 

Teachers*  Perception  of 
Resource  Teacher 

% 

Resource  Teacher’s  Perception 
of  her/his  Role 

School 

• Provides  computer 

36.6 

• Show  teachers  what  technologies  we 

6 

workshops/Trains  teachers  and 

have  and  how  they  can  utilize  them 

provides  enrichment/Trains 

• Convince  teachers  that  they  save 

teachers  in  technology 

time  by  using  computers 

• Teacher  of  teachers/ Assistant  to 

27 

• Lots  of  troubleshooting 

teachers  and  students 

• Make  teachers  feel  comfortable  with 

• Troubleshooter/keeps  computers 

23 

computers  and  with  their  computer 

in  working  order 

skills 

• Facilitator/Rescuer 

16.6 

• Make  teachers  comfortable  to  bring 

• Helps  plan  and  implement 

16.6 

their  students  to  computer  lab 

computer  lab  lessons/Con sultant 

for  class  activities/Teaching  or 

introducing  new  skills  to  students 

• Helps  teach  lesson s/Helps  teach 

classes  and  answer  questions 

16.6 

• Install  and  maintain  computer 

programs/software 

6.7 

• Too  vast  to  mention/Is  other  half 

of  my  brain 

6.6 

• Keeps  us  abreast  of  technology 

• Assist  with  technical  problems 

6.6 

• Is  omniscient  in  computer  lab 

6.6 

• Aids  in  the  use  of  computer  for 

3.3 

instruction 

3.3 

Table  5 

Summary  of  teachers’  responses  to  the  question  "In  what  ways  TRT  help  you  with  computer  usage?." 


Categories 
of  Responses 

School 
# 1 

School 

#2 

School 

#3 

School 

#4 

School 

#5 

School 

#6 

Average 

1 . Finding  proper  software/ 
updating  software. 

34.8% 

(8) 

n.i% 

(3) 

16.7% 

(3) 

3.2% 

(1) 

16.7% 

(8) 

13.3% 

(4) 

16.0% 

2.  Available  all  the  time 
to  help  with  what  we 
want. 

26.1 

(6) 

7.4% 

(2) 

16.7% 

(3) 

16.1% 

(5) 

4.2% 

(2) 

26.7% 

(8) 

16.2% 

3.  Teaching  how  to  use 
computer/teaching 
computer  competency 
test. 

13.0% 

(3) 

11.1% 

(3) 

16.7% 

(3) 

19.4% 

(6) 

4.2% 

(2) 

26.7% 

(8) 

15.2% 

4.  Helping  through  staff 
development  and 
workshops. 

17.4% 

(4) 

18.5% 

(5) 

16.7% 

(3) 

6.5% 

(2) 

10.4% 

(5) 

16.7% 

(5) 

14.4% 

5.  Trouble  shooting 

13.0% 

(3) 

0.0% 

(0) 

22.2% 
(4)  _ 

16.1 

(5) 

8.3% 

(4) 

16.7% 

(5) 

12.7% 

6.  Answering  question 
about  technology. 

26.1% 

(6) 

7.4% 

(2) 

0.0% 

(0) 

12.9% 

(4) 

4.2% 

(2) 

16.7% 

(5) 

11.2% 

Total  number  of 
observations 

23 

27 

18 

31 

r 

30 

O 
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Table  6 

Resource  Teachers’  versus  Teachers’  Perceptions  of  TRT  Role 


Job  Description 
Role  and  Responsibility 

Technology  Coordinators’ 
Perception 

Principals’  Perception 

Instructional  8 phases 

Technical  4 phases 

Administrative  5 phases 

Instructional 

•teach  introductory  and  computer 
related  lesson 

•assist  teachers  in  selecting  and  using 
materials  and  equipment  suitable  for 
grade  level 

•update  computer  curriculum 

•assist  teachers  in  integrating  computer 

activities 

•assist  teachers  in  implementing  special 
projects  with  multimedia,  publications, 
telecommmunications 
•teaching  demonstration  lessons  with 
more  software 

•assist  teachers  with  understanding  of 
computers 

•responsible  for  helping  teachers  pass 
competency  test 

Technical 

•supply  technical  expertise 
•troubleshooting  computers,  printers, 
etc, 

•serve  as  consultant  for  purchasing 
equipment 

•install  new  equipment 

Administrative 

•distribute  copies  of  items  and  state 
licenses 

•distribute  documentation  of  district 
owned  and  licenses  software 
•maintain  a schedule  for  computer  lab 
•work  with  media  specialist  to  keep 
inventory 

•work  with  technology  department  as  a 
member 

Qualifications 

•knowlege  in  the  application  of 
instructional  technology 
•experience  in  working  with  teachers 
•at  least  three  years  teaching 
experience 

•we  looked  at  technical  skills 
and  hardware  and  software 
knowledge 

•we  really  concentrated  a lot  on 
skills  of  working  with  DOS,  to 
do  training  with  DOS  which 
would  be  intimidating  to  adults 

•we  did  not  focus  on  years  of 
teaching  (3  years  teachings  is 
one  of  the  job  qualifications  for 
the  position) 

•we  just  want  somebody  who 
has  solid  background  with  good 
skills  in  technology 

•they  need  to  be  very  personable 
and  be  able  to  work  with  people 

•they  should  have  had  some 
experience  in  training  adults 

•I  would  want  somebody  very 
comfortable  in  technology 

•Technical  knowledge 

•Someone  who  can  interface 
with  people  and  has  good  people 
skills 

•Human  rapport  and 
understanding  of  teachers 

•Someone  in-house  on  faculty 
so  transition  would  be  easier 

, •Warm  and  nuturing 
personality,  not  intimidating 

•Committed  to  staff  and 
students 

•A  teacher  and  facilitator 
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Table  7 

Summary  of  teachers’  responses  to  the  question  "Describe  how  you  use  the  computer  in  your  classroom  to  aid  students 
learning  within  the  subject  area  you  teach." 


Categories  of 
Responses 

School 
# 1 

School 

#2 

School 

#3 

School 

#4 

School 

#5 

School 

#6 

Average 

1 . Use  subject  matter 
related  software 

21.7% 

(5) 

3.7% 

(1) 

22.2% 

(4) 

22.6% 

1(7) 

27.1% 

(13) 

26.7% 

(8) 

21.0% 

2.  Use  word-processing 
software 

34.8% 

(8) 

14.8% 

(4) 

22.2% 

(4) 

29.0% 

(9) 

10.4% 

(5) 

10.0% 

(3) 

20.2% 

3.  Use  for  remediation* 
enrichment 

43.5% 

(10) 

3.7% 

(1) 

11.1% 

(2) 

16.1% 

_£! 

10.4% 

(5) 

13.3% 

(4) 

16.4% 

4.  Use  for  keyboarding 

13.0% 

(3) 

7.4% 

(2) 

0.0% 

(0) 

3.2% 

m 

0.0% 

(0) 

6.7% 

(2) 

5.1% 

5.  Use  for  lesson 
planning/tests/ 
transparencies 

0.0% 

(0) 

3.7% 

(1) 

5.6% 

(1) 

6.5% 

(2) 

6.3% 

(3) 

6.7% 

(2) 

4.8% 

6.  use  games  for. 
reinforcement/ 
motivation. 

4.4% 

(1) 

7.4% 

(2) 

0.0% 

(0) 

3.2 

(1) 

2.1% 

(1) 

6.7% 

(2) 

4.0% 

Total  number  of 
observations 

23 

27 

18 

31 

48 

30 

Table  8 

Cross  tabulation  of  classroom  computer  and  its  usage  by  school. 


School 


Questions 

i 

2 

3 

4 

5 

6 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

(n=17) 

Do  you  have  computer  in 

17.4 

16.3 

15.1 

19.8 

15.1 

16.3 

your  classroom? 

(15) 

(14) 

(13) 

(17) 

m 

(14) 

Is  your  classroom  computer 
compatible  with  the  other 
computers  in  your  school  and 
with  the  computer  lab? 

18.8 

16.3 

15.0 

20.0 

15.0 

15.0 

Do  you  have  any  software  in 
your  classroom? 

16.9 

18.3 

18.3 

15.5 

14.1 

16.9 

Which  of  the  following 
describes  how  you  are  using 
your  classroom  computer  and 
educational  software? 

• Design  visual 
presentation 

8.3 

8.3 

12.5 

16.7 

25.0 

29.2 

• Prepare  hand-outs  & 
print 

15.1 

18.9 

17.0 

17.0 

15.1 

17.0 

• For  remediation 

28.1 

15.6 

18.8 

6.3 

18.8 

12.5 

• Enhance  learning 

20.0 

14.0 

18.0 

16.0 

16.0 

16.0 
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Table  9 

Teachers’  responses  to  the  question  "Why  have  you  selected  certain  areas  as  your  weakest  areas?." 


Categories  of 
Responses 

School 
# 1 

School 

#2 

School 
# 3 

School  ; 

#4 

School 

#5 

School 

#6 

Average 

1.  Lack  of 
experience. 

30.4% 

(7) 

33.3% 

(9) 

27.8% 

(5) 

3.2% 

(1) 

45.8% 

(22) 

56.7% 

(17) 

32.9% 

2.  Need  more 
instruction. 

43.5% 

(10) 

3.7% 

(1) 

27.8% 

15) 

3.2% 

ID , 

31.3% 

m 

16.7% 

J2 

21.0% 

3.  Lack  of  facilities. 

0.0% 
(0)  ... 

0.0% 

(0) 

3.7% 

(1) 

0.0% 

(0) 

0.0% 

(0) 

0.0% 

(0) 

3.7% 

Total  number  of 
observations 

23 

27 

18 

31 

48 

30 

Table  10 

Results  of  background  questions  in  the  Computer  Attitude  Survey  before  equalizing  the  sample  size  across  schools. 


Questions Responses 


L What  grade  level  do  you  teach? 

6th 

(43) 

21% 

7th 

(46) 

22% 

8th 

(45) 

21% 

Others 

(62) 

35% 

2.  What  subject  areas  do  you  teach? 

Math 

Science 

LA 

Social 

Other 

Studies 

(26) 

(19) 

(38) 

(14) 

(109) 

12.6% 

9.2% 

18.4% 

6.8% 

52 

3.  Are  you  seeking  credit  for  computer 

Yes 

No 

competency  workshop  that  you 

(160) 

(39) 

take? 

77.7% 

18.9% 

4.  How  many  hours  of  computer 

>10 

10-20 

20-30 

30-40 

workshop  training  have  you 

(45) 

(74) 

(36) 

(50) 

completed? 

21.8% 

35.9% 

17.5% 

24.3% 

5.  Are  you  currently  using  computers 

Yes 

No 

in  your  classroom? 

(168) 

(37) 

81.6% 

18% 

6.  Before  this  year,  what  previous 

0 

Few 

1-6  M 

7 M 

Over  1 Y 

experience  have  you  had  in 

days  to 

tol  Y 

operating  computers? 

1 M 

(23) 

(32) 

(17) 

(16) 

(118) 

11.2% 

15.5% 

8.3% 

7.8% 

57.3% 

7.  Describe  your  home  computer. 

Mac 

IBM 

Apple 

other 

None 

(10) 

(92) 

(5) 

08) 

(75) 

4.9% 

44.7% 

2.4% 

8.7% 

36.4% 

8.  How  would  you  rate  your  computer 

None 

Poor 

Average 

Good 

Excellent 

skills  before  the  school  year 

started? 

(19) 

(63) 

(69) 

(42) 

(13) 

9.2% 

30.7% 

33.5% 

20.4% 

6.3% 

9.  How  would  you  rate  your  ability  to 

None 

Poor 

Average 

Good 

Excellent 

use  computer  now? 

(5) 

(35) 

(78) 

(64) 

(24) 

2.4% 

17.0% 

37.9% 

31.1% 

11.7% 
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Table  11 

Correlation  between  liking  computers  and  other  variables 


Variables 

N 

R 

Liking  computers  vs.  value  for  computers 

133 

.2851* 

Liking  computers  vs.  confidence  for  learning  computers 

134 

.6542* 

Liking  computers  vs.  confidence  for  using  computers 

134 

.6569* 

Liking  computers  vs.  value  for  computers  in  education 
Liking  computers  vs.  rate  computer  skills  before 

133 

.2851* 

this  year 

135 

.4760* 

Liking  computers  vs.  rate  using  computers  now 

135 

.5024* 

Liking  computers  vs.  previous  experience  with  operating 

135 

.3725* 

Liking  computers  vs.  current  usage  of  computer  in  class 

135 

.2482* 

*P<.001 


Table  12 

Correlation  between  confidence  for  learning  computers  and  the  other  variables 


Variables  N 


Confidence  for  learning  computers  vs.  value  for  using  computers 
Confidence  for  using  computers  vs.  confidence  for  learning 

141 

.6534* 

computers 

Confidence  for  learning  computers  vs.  rate  computer  skills  before 

141 

.6534* 

this  year 

Confidence  for  learning  computers  and  rate  ability  using 

142 

.3823* 

computers  now 

Confidence  for  learning  computers  vs.  previous  experience  with 

142 

.5056* 

computers 

204 

.2652* 

*p<.005 

Table  13 

Correlation  between  value  for  computers  in  education  and  other  variables 

Variables 

N 

R 

Confidence  for  using  computers  vs.  previous  experience  with 
computers 

142 

.5050* 

Confidence  for  using  computers  vs.  rate  computer  skills  before 
this  year 

Confidence  for  using  computers  vs.  rate  ability  using  computers 

142 

.6320* 

now 

142 

.6400* 

Confidence  using  computers  vs.  hours  of  computer  competency 
workshop 

141 

.3001* 

*p<.001 
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Table  14 

Correlation  between  anxiety  for  computer  and  the  other  variables 


Variables  N R 


Anxiety  for  computer  vs.  liking  computers 

134 

.5486* 

Anxiety  for  computers  vs.  confidence  for  learning  computers 

141 

.5928* 

Anxiety  for  computers  vs.  confidence  for  using  computers 

142 

.7685* 

Anxiety  for  computers  vs.  rate  computer  skills  before  this  year 

142 

.5758* 

Anxiety  for  computer  vs.  rate  ability  to  use  computers  now 

142 

.5486* 

*p<.001 


Table  15 

Correlation  between  rate  of  computer  skills  and  other  variables 
Variables 

N 

R 

Rate  of  computer  skills  vs.  hours  of  computer  competency 

143 

.2642* 

workshops 

Rate  computer  skills  before  this  years  vs.  rate  ability  to  use 

computers  now 

144 

.7269* 

Rate  ability  to  use  computer  vs.  hours  of  computer  competency 

workshops 

143 

.3229* 

*p<.001 

Table'  16: 

Correlation  among  variables  in  general  survey. 

Variables 

R 

P value 

Home  computer  vs.  computer  skills  at  present 

.3007 

.01 

Home  computer  vs.  computer  skills  prior  to  training 

.2630 

.01 

Home  computer  vs.  home  computer  usage 

.5372 

.01 

Home  computer  usage  vs.  computer  skills  prior  to  training 

.2543 

.05 

Home  computer  usage  vs.  computer  skills  at  present 

.2605 

.05 

Computer  skills  prior  to  training  vs.  computer  skills  at  present 

.6856 

.01 

Computer  skills  at  present  vs.  hours  of  computer  workshop 

.3930 

.001 
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Table  17 

Descriptive  statistics  for  teachers'  attitude  toward  computer  by  six  different  schools 


School 

School 

School 

School 

School 

School 

1 

2 

3 

4 

5 

6 

(n=24) 

(n=24) 

(n=24) 

(n=24) 

(n=24) 

(n=24) 

Factors  measured  teachers' 

M 

M 

M 

M 

M 

M 

attitude  toward  computers 

(SD) 

(SD) 

(SD) 

(SD) 

(SD) 

(SD) 

1. 

Liking  computers 

24.2 

25.6 

22.9 

25.6 

21.1 

24.2 

(4.8) 

(4.9) 

(5.7) 

(8.2) 

(6.5) 

(6.7) 

2. 

Value  for  computer  in 

49.5 

49.5 

50.0 

50.7 

48.8 

48.4 

education 

(4.6) 

(3.6) 

(2.8) 

(4.0) 

(6.5) 

(8.7) 

3. 

Confidence  for  learning 

3.6 

3.3 

3.1 

3.6 

2.7 

3.1 

computers 

(1.0) 

(1.2) 

(1.2) 

(1.7) 

(1.1) 

(1.1) 

4. 

Confidence  for  using 

4.1 

3.8 

3.4 

4.4 

3.8 

4.1 

(1.4) 

(1.7) 

(1.3) 

(2.1) 

(2.5) 

(2.1) 

5. 

Anxiety  for  computers 

28.1 

28.5  30.8 

26.5 

27.3 

27.6 

(4.8) 

(5.3) 

(4.3) 

(7.4) 

(7.3) 

(7.7) 

Table  18 

Descriptive  statistics  for  teachers'  attitude  toward  computer  by  those  who  are 
currently  suing  computers  and  those  who  are  not  currently  using  computers. 


Using  computers  Not  Using  computers 


Factors  measured  teachers' 
attitude  toward  computers 

M 

SD 

M 

SD 

1 . Liking  computers 

23.2 

5.8 

28.1 

7.9 

2.  Value  for  computer  in 

49.1 

5.4 

52.6 

4.6 

education 

3 . Confidence  for  learning 

3.2 

1.2 

3.5 

1.6 

computers 

4.  Confidence  for  using 

3.8 

1.8 

4.5 

2.1 

computers 

5 . Anxiety  for  computers 

28.4 

6.1 

26.6 

7.5 
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Table  19 

Descriptive  statistics  for  teachers'  attitude  toward  computer  by  previous  experience  in  operating  computers. 


0 experience  Few  days  1 to  6 months  7 month  Over  a year 


Factors  measured  teachers' 
attitude  toward  computers 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

1.  Liking  computers 

21.9 

23.2 

24.8 

27.2 

30.7 

(6.4) 

(7.6) 

(4.7) 

(4.0) 

(5.3) 

2.  Value  for  computer  in 

103.0 

102.7 

104.8 

104.7 

105.5 

education 

(6.2) 

(4.1) 

(3.0) 

(3.8) 

(6.9) 

3 . Confidence  for  learning 

2.9 

3.5 

3.3 

3.7 

3.9 

computers 

(1.0) 

(1.3) 

(1.5) 

(1.7) 

(1.1) 

4.  Confidence  for  using 

3.3 

3.9 

4.0 

0.0 

0.0 

computers 

(1.5) 

(1.3) 

(12) 

(0.0) 

(0.0) 

5 . Anxiety  for  computers 

20.8 

28.2 

28.8 

25.9 

20.7 

(5.8) 

(3.4) 

(3.6) 

(6.1) 

(7.0) 

Descriptive  statistics  for  teachers'  attitude  toward  computer  by  rate  computer  skills  before  school  year. 

None  Poor  Average  Good  Excellent 


Factors  measured  teachers' 
attitude  toward  computers 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

1 . Liking  computers 

19.1 

19.1 

24.1 

26.7 

30.2 

(5.3) 

(3.9) 

(5.8) 

(5.8) 

(5.8) 

2.  Value  for  computer  in 

104.2 

102.2 

103.4 

104.1 

106.2 

education 

(2.8) 

(6.0) 

(6.7) 

(3.6) 

(6.8) 

3.  Confidence  for  learning 

2.3 

2.6 

3.3 

3.8 

3.8 

computers 

(10) 

(0.7) 

(1.1) 

(1-4) 

(1.3) 

4.  Confidence  for  using 

2.0 

2.7 

3.7 

5.1 

6.1 

computers 

(0.0) 

(1.2) 

(1.2) 

(1.8) 

(2.3) 

5 . Anxiety  for  computers 

34.4 

31.3 

29.2 

24.9 

20.7 

(1.3) 

(5.1) 

(4.7) 

(5.8) 

(7.4) 

Table  21:  Descriptive  statistics  for  teachers'  attitude  toward  computer  by  rate  ability  to 
use  computer  now. 


Factors  measured  teachers' 
attitude  toward  computers 

None 

Poor 

Average 

Good 

Excellent 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

M 

(SD) 

1 . Liking  computers 

17.9 

21.6 

25.7 

30.3 

22.0 

(S.?) 

(4.6) 

(5.7) 

(6.2) 

(2.8) 

2.  Value  for  computer  in 

102.0 

103.5 

104.0 

104.7 

101.5 

education 

(6.6) 

(4.3) 

(6.5) 

(5.2) 

(3.5) 

3.  Confidence  for  learning 

2.1 

2.7 

3.6 

4.5 

2.0 

computers 

(0.3) 

(0.8) 

(10) 

(15) 

(0.0) 

4 . Confidence  for  using 

2.2 

3.0 

4.3 

6.6 

2.5 

computers 

(0.6) 

(1.1) 

(1.3) 

(1-8) 

(0.7) 

5 . Anxiety  for  computers 

32.7 

30.8 

27.0 

19.8 

34.5 

(4.9) 

(4.3) 

(4.5) 

(6.5) 

(0-7) 
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Table  22 

Multivariate  analysis  of  variance  for  teachers  attitude  toward  computers  school,  usage  of  computer,  previous  experience 
and  rate  of  computer  skills. 


Univariate  F Ratio 


Multivariate  Tests  of  Significance  Factors  measuring  teachers'  attitude  toward  computers 


Source 

F 

1 

2 

3 

4 5 

1 . Attitude  by  school 

0.78 

1.6 

0.52 

1.59 

0.70  1.24 

2.  Attitude  by  usage  of  computer 

0.88* 

10.45* 

7.93** 

0.733 

2.27  1.39 

3.  Attitude  by  previous  computer 

0.62*** 

9.4*** 

0.62 

2.9 

12.1**  8.3*** 

experience 

4.  Attitude  by  rate  computer 

0.51*** 

12.6*** 

1.0 

6.5*** 

19.9***  15.6*** 

skills  before  school  year 

5.  Attitude  by  rate  ability  to  use 

0.40*** 

17.0*** 

0.69 

16.6*** 

35.3***  22.3*** 

computer  now 

Note:  Analyses  are  on  (1)  liking  computers,  (2)  value  for  computer  in  education,  (3)  confidence 
learning  computers,  (4)  confidence  using  computer,  and  (5)  anxiety  for  computers. 
ldf=5,  124  2df=  1,1283  df=  1,  125  4df=4,  125 

* p<.05  **p<01  ***p<.001 
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This  paper  reflects  the  contributions  of  five  professors  and  one  doctoral  student  of  Educational  Technology,  who 
serve  as  panelists  for  the  13th  Annual  Foundations  Symposium  and  Open  Forum.  This  year’s  theme  is  Responding  to 
Cultural  Diversity , and  continues  the  tradition  of  presenting  topics  and  issues  that  are  of  current  relevance  to  the  field,  and 
are  foundational  in  nature.  The  issue  at  hand  — the  impact  of  learner  diversity  on  the  practice  of  Educational  Technology 
— is  one  which  extends  the  current  boundaries  of  the  field  and  fosters  dialog  between  and  among  the  presenters  and 
audience. 

Both  this  manuscript  and  open  forum  extend  the  present  knowledge  base  in  the  field  by  addressing  key  critical 
issues  regarding  learner  diversity.  The  body  of  this  document  is  presented  in  six  parts:  1)  Powell  begins  by  asking  why 
address  diversity,  why  should  the  field  listen,  what  the  benefits  are,  and  for  whom;  2)  DeVaney  questions  why  culture  has 
not  been  a part  of  the  Educational  Technology  discourse;  3)  Branch  examines  mechanisms  for  utilizing  existing 
frameworks  for  incorporating  cultural  pluralism  into  instruction;  4)  Knupfer  reveals  the  impact  gender  has  on  the  design 
of  instruction;  5)  Thompson  asserts  that  cultural  diversity  must  be  explicitly  addressed  within  current  ID  models;  and  6) 
Reeves  reflects  on  personal  experiences  evaluating  culturally  insensitive  materials. 
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(1)  Responding  to  Cultural  Diversity:  Who,  Why,  and  For  What? 

Gary  C.  Powell,  Ed.D. 

Why  address  diversity?  The  rhetoric  of  educational  technology  (as  depicted  in  design  models  and  development 
guides)  is  very  clear  that  learner  characteristics  such  as  prior  knowledge,  entry  behaviors,  ability  and  motivation  must  be 
taken  into  consideration  to  increase  student  learning.  Teachers  and  designers  both  are  urged  to  recognize  those 
psychological  characteristics  that  set  one  student  off  from  another.  Instruction  is  more  effective  when  it  is  fitted  to  a 
learner's  uniqueness  as  a person.  Such  matters  as  mental  abilities,  aptitudes,  and  motivations  are  thus  regarded  as 
individual  endowments  worthy  of  respect  and  available  for  use  in  the  learning  process.  Matters  of  cultural,  racial  and 
ethnic  diversity,  however,  have  not  elicited  similar  treatments  in  the  educational  technology  literature,  including  our 
texts,  journals,  and  conference  presentations.  No  longer  can  instructional  technology  ignore  the  critical  challenges  to 
educating  diverse  students.  Preparing  learners  to  function  in  an  ever-changing  international  marketplace  requires 
paradigmatic  shifts  in  the  art  and  science  of  educational  technology. 

Why  should  l listen?  There  is  strong  support,  particularly  from  researchers  and  practitioners  in  other  fields  of 
Education,  that  learners’  perceptions  of  the  educational  process  depends  largely  on  the  values,  attitudes,  and  behaviors  of 
those  within  their  cultural  group.  All  learners  therefore  bring  culturally  based  rules,  expectations,  value  systems,  and 
needs  about  education  to  the  learning  environment.  Increasing  our  ability  to  understand  them,  respond  to  then?,  and 
accommodate  for  them  within  the  current  Instructional  Systems  Design  framework,  takes  us  one  step  closer  to  education 
which  is  meaningful,  relevant  and  effective  for  all.  4 All’  includes  persons  who  may  not  look  or  sound  like  us,  and  who’s 
preferred  learning  and  teaching  styles  may  not  be  the  same  as  ours  either.  It  is  critical  that  we  as  educators  value  these 
alternative  styles,  and  see  them  as  viable  and  valid  assets  to  instruction.  We  should  not  imposing  practices,  orientations 
or  expectations  which  reflect  our  own  (often  mainstream)  culture  on  others. 

How  shall  / benefit  by  responding  to  diversity?  Good  question!  Perhaps  a better  question  should  be  how  shall 
my  learning  audience  benefit,  because  your  benefit  (what's  in  it  for  me)  will  stem  from  that  of  the  learners.  In  other 
words,  as  designers,  we  experience  satisfaction  when  we  accurately  and  consistently  meet  the  psychological  and 
motivational  needs  of  the  populations  for  whom  we  design  instruction.  We  faithfully  follow  our  models,  and  focus  on 
individual  characteristics  (entry  behaviors)  such  as  motivation,  age,  achievement  level,  cognitive  style  and  reading  level. 
The  end  result  (at  times)  is  Effective’  instruction;  at  which  point  we  are  happy.  While  important  to  address,  such 
traditional  individual  characteristics  miss  other  salient  forms  of  diversity,  especially  cultural.  The  cultural  background  of 
the  learner  must  be  determined  and  explicitly  addressed.  Basic  learner  analysis  and  user-centered  designs  are  not  enough, 
because  like  it  or  not,  designers’  instructional  ‘creations’  are  shaped  by  their  own  culturally  influenced  values,  norms, 
beliefs  and  morals. 

How  will  you  benefit  by  responding  to  diversity?  First,  by  recognizing  that  you  now  have  adequately  analyzed 
your  learning  audience  (which  in  all  contexts  grows  more  diverse  each  day).  And  second,  from  the  satisfaction  of 
successfully  incorporating  a culturally  pluralistic  focus  into  the  design  of  instruction,  such  that  it  capitalizes  on  learners’ 
distinctive  learning  styles,  strengths  and  orientations. 
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Who  am  I?!  You  are  an  instructional  technologist,  instructional  designer,  needs  assessment  expert,  evaluation 
expert,  media  specialist,  curriculum  specialist,  teacher,  administrator,  professor,  HRD  expert,  training  manager  or  trainer 
who  wishes  to  deliver  instruction  which  is  responsive  to  the  needs  of  diverse  learners. 
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(2)  Negligent  Discourse 
Ann  DeVaney,  Ph.D. 

As  the  field  of  educational  technology  emerged  from  the  audiovisual  area,  scholars  and  practitioners  yearned  for  a 
neutral  discourse  that  would  bring  rigor  and  respect  to  the  field.  Informed  by  behavioral  theory  and  systems  models,  a 
dominant  discourse  did  coalesce  (DeVaney  and  Butler,  1996).  It  claimed  to  be  culture  free  and  no  communications  about 
race  and  gender  appeared  for  decades.  Can  these  conversations  now  take  place? 

A discourse  is  an  informal  system  of  thought  that  is  apparent  in  the  rhetoric  of  scholarship,  speech  and 
pedagogy  within  a discipline.  You  can't  pin  it  down  and  dissect  it,  but  it  is  there  and  its  material  effects  are  felt  (Bove, 
1992).  Some  people  believe  that  the  Oklahoma  City  bombing  was  the  material  effect  of  the  rhetoric  of  a specific 
discourse.  The  morning  after  the  Iowa  caucuses.  Bob  Dole  adopted  the  rhetoric  of  a specific  form  of  populism  that  had 
garnered  many  votes  for  Pat  Buchanan.  The  material  effects  of  a discourse  that  excludes  talk  about  race  may  be  fewer 
jobs  for  people  of  color  in  the  field  that  espouses  the  discourse. 

Although  many  discourses  inform  scholarship  in  educational  technology,  there  has  always  been  a dominant  one, 
a sanctioned  discourse  that  forms  official  knowledge  in  the  field.  (Apple,  82)  Whether  it  was  audiovisual  research  in  the 
20s,  30s,  or  40s;  or  instructional  technology  in  the  50s,  or  60s;  or  instructional  systems  technology  in  the  70s  and  80s, 
a hallmark  of  the  official  knowledge  was  its  valorization  of  psychology,  the  study  of  the  brain,  a unit  of  one.  Whether 
connectionism,  operant  conditioning,  information  processing  or  schema  theory,  educational  technology  scholars  used 
models  of  brain  functions  to  describe  the  relationship  between  machines  and  learning,  or  software  design  and  learning. 

The  field  appeared  inextricably  linked  to  the  field  of  educational  psychology,  so  much  so  that  it  was  hard  for  practitioners 
to  conceive  of  learning  in  terms  more  expansive  then  the  explanation  offered  by  a unit  of  one.  (DeVaney  and  Butler, 

1996)  From  Immanuel  Kant  to  the  philosophers  of  the  Vienna  Circle,  the  concept  of  the  mind  and  brain  has  been  boxed 
and  delivered  to  educators  in  the  rhetoric  of  mathematics  and  physics  (Rundell,  1996). 

Not  to  belabor  the  point,  but  we  have  suffered  under  the  tyranny  of  psychology.  The  discourse  used  to  describe 
human  learning  was  devoid  of  concepts  of  culture,  yet  we  all  recognize  that  we  only  learn  some  things  because  of  our 
membership  in  a group.  We  recognize  that  meanings  are  negotiated.  The  frequent  iteration  of  certain  concepts,  theories, 
definitions,  assumptions  and  values  has  created  a psychological  discourse  that  shackled  our  thinking  about  learning;  that 
shackled  our  epistemologies. 

Some  postmodern  discourses  break  the  shackles  of  reductive  psychological  notions  (Derrida,  70,  76,  78; 

Foucault,  79;  Rorty,  82,  91).  Yet  of  all  the  80s  and  90s  scholarship  in  schools  of  education,  why  is  ours  the  last  to 
consider  issues  of  culture,  issues  of  race,  gender  and  ethnicity.  Rather  than  answer  that  question  directly,  I'd  like  to  say 
that  the  time  is  here  for  discussion  of  educational  technology  and  cultural  pluralism.  And,  I'd  like  to  ask  you  to  listen  and 
try  to  identify  what  discourses  inform  the  messages  of  ^»ur  speakers  today  on  cultural  pluralism. 

References 

Apple,  M.  (1982).  Education  and  Power,  London;  Routledge  and  Kegan  Paul. 

Bove,  M.  (1992).  Mastering  Discourse,  Durham  , North  Carolina;  Duke  University  Press. 

Denida,  J.  (1970).  Structure,  sign  and  play  in  the  discourse  of  the  human  sciences,  The  structuralist  controversy,  ed.  R. 

Macksey  and  E.  Donato,  Baltimore:  The  Johns  Hopkins  University  Press,  247-265. 

Derrida,  J.  (1976).  Of  grammatology.  tr.  G.  Chakravorty  Spivak.  Baltimore:  Ihe  Johns  Hopkins  University  Press. 
Derrida,  J.  (1978).  Writing  and  difference.  A Bass,  London:  Routledge  and  Kegan  Paul. 

DeVaney,  A & R.  Butler  (1996).  Voices  of  the  Founders;  Discourses  in  Early  Educational  technology  in  Handbook  of 
Research  on  Educational  Communications  & Technology.  New  York:  Scholastic  Publishing.  (Forthcoming) 
Foucault,  M.  (1979).  Discipline  and  Punish;  The  Birth  of  the  Prison,  New  York;  Vintage  Books. 

Rorty,  R.  (1982).  Consequences  of  Pragmatism.  Minneapolis,  MN:  University  of  Minnesota  Press. 

Rorty,  R.  (1991).  Essays  on  Heidegger  and  Others.  New  York:  Cambridge  University  Press. 


r: 


S4 


550 


os 

(3)  Utilizing  Existing  Frameworks  for  Incorporating  Cultural  Pluralism  into  Instruction 

Robert  Maribe  Branch,  Ed.D. 

Instructional  technologists  have  tools  available  to  respond  to  the  current  demand  for  planning  instruction  which 
is  culturally  sensitive.  The  domains  of  instructional  technology  (Seels  & Richey,  1994)  offer  a conceptual  framework 
for  understanding  the  relationship  between  research,  theory  and  practice  of  inclusive  educational  communications. 
Systematic  designs  of  instruction  (Andrews  & Goodson,  1980;  Edmonds,  Branch  & Mukheijee,  1994)  provide  procedural 
frameworks  for  identifying  opportunities  to  assure  an  appropriate  match  between  abilities,  values  and  perceptions  of  the 
intended  learner  audience,  and  negotiated  learner  outcomes.  The  common  events  associated  with  an  instructional  episode 
(Gagn6,  Briggs  & Wager,  1992)  suggests  a pedagogical  framework  for  accomplishing  exchanges  where  learners 
deconstruct  and  construct  knowledge  through  cultural  perspective.  The  purpose  here  is  to  recommend  ways  in  which 
instructional  designers  can  routinely  incorporate  cultural  pluralism  into  episodes  of  instruction. 

Pluralism,  rather  than  diversity,  is  the  term  of  choice  because  cultural  diversity  exists  within  any  learner 
audience  regardless  of  whether  or  not  the  quality  or  quantity  of  the  diversity  is  acknowledged  by  the  instructional  designer. 
Cultural  pluralism  requires  action  in  order  to  realize  any  learner  potential.  Therefore,  diversity  is  merely  a recognition 
state,  which  is  passive;  while  pluralism  is  an  active  state  requiring  purposeful  activities  designed  to  elicit  the  uniqueness 
of  the  individuals  that  form  the  group.  Thus,  diversity  of  the  learner  audience  emerges  as  positive  attributes  upon  which 
an  instructional  episode  can  be  constructed. 

The  domains  of  instructional  technology:  design , development ; evaluation,  management  and  diffusion  (Seels  & 
Richey,  1994),  conceive  categorical  notions  about  how  to  research,  theorize  and  practice  instruction.  Each  domain  also 
offers  an  opportunity  to  incorporate  culture  as  part  of  an  educational  communication.  Design  offers  the  opportunity  to 
scrutinize  the  origin  of  content  and  other  knowledge  structures  in  order  to  determine  the  validity  and  meaningfulness  of 
what  is  to  be  taught.  Development  offers  opportunities  to  study  the  impact  inclusive  ideas  will  have  on  an  ability  to 
deliver  those  ideas.  Evaluation  offers  opportunities  to  explicitly  assess  the  value  of  instructional  content  relative  to  the 
values  of  the  learner  audience.  Management  offers  opportunities  to  insure  issues  of  equity  and  access  are  addressed  among 
organizations  and  institutions  associated  with  education,  training  and  the  development  of  human  resources.  Diffusion  of 
innovations  offer  opportunities  to  extend  ideas  about  the  value  of  diversity  and  pluralism  beyond  surface  levels  to  deep 
levels  so  that  incorporating  cultural  pluralism  becomes  a way  of  life.  While  the  domains  of  instructional  technology 
conceptualize  opportunities  for  culturally  pluralistic  instruction,  the  systematic  design  of  instruction  provides  procedures 
for  realizing  the  research,  theory  and  practice  conceived  among  the  domains. 

Instructional  design  that  is  systemic,  systematic,  based  on  genera!  systems  theory,  and  adopts  an  input-process- 
output  paradigm  provides  procedures  for  incorporating  cultural  pluralism  into  instruction.  The  components  of  a systems 
approach  model:  determine  instructional  goal,  analyze  the  instructional  goal,  analyze  learners  and  contexts,  wriis 
performance  objectives , develop  assessment  instruments,  develop  instructional  strategy,  develop  and  select  instruction, 
design  and  conduct  the  formative  evaluation  of  instruction,  revise  instruction  and  conduct  summative  evaluation  (Dick  & 
Carey,  1996),  can  independently  facilitate  ways  to  assure  instructional  designers  that  cultural  aspects  are  consistently 
incorporated  into  any  instruction. 

Determine  instructional  goal  provides  a procedure  where  learner  expectations  can  originate  from  analyses  of 
people  already  doing  a job  and  are  similar  in  cultural  background  as  members  of  the  learner  audience.  Analyze  the 
instructional  goal  provides  a procedure  where  a diagram  is  generated  that  illustrates  the  diverse  skills  and  knowledge 
required  for  a learner  to  achieve  a goal.  Analyze  learners  and  contexts  provides  a procedure  where  individual  learners  and 
grouped  learner  audiences  can  be  understood  in  terms  of  their  preferences,  skills  and  attitudes  that  are  culturally  influenced. 
Write  performance  objectives  provides  a procedure  where  specific  expectations  of  the  learner  can  be  matched  directly  with 
the  cultural  values  and  learning  styles  of  the  learner.  Develop  instructional  strategy  provides  a procedure  where  each 
event  of  an  instructional  episode  can  correlate  with  activities  relevant  to  out-of-school  experiences  of  the  learner  audience 
which  support  knowledge  construction  and  skill  acquisition.  Develop  and  select  instruction  provides  a procedure  where 
instructional  media  is  utilized  to  portray  images  and  discourse  similar  to  the  learner  audience.  Design  and  conduct  the 
formative  evaluation  of  instruction  provides  a procedure  where  members  of  the  intended  audience  directly  inform  the 
instructional  improvement  process  by  adding  his  or  her  own  perspective  about  any  planned  instructional  episodes. 

Revise  instruction  provides  a procedure  where  each  person  can  add  dava,  information,  knowledge  and  wisdom  to  the 
planned  instruction  based  on  personal  cultural  perspectives.  Conduct  summative  evaluation  provides  a procedure  where 
the  value  and  worth  of  any  instruction  can  be  subjected  to  the  critical  judgment  of  all  stakeholders  in  terms  of  cultural 
sensitivity.  While  instructional  design  provides  a framework  of  procedures  for  incorporating  cultural  pluralism  into 
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instruction,  the  sequence  of  events  which  actually  form  the  instructional  episode  depend  upon  a pedagogical  framework 
that  supports  categorical  inclusions  of  cultural  perspectives. 

Instructional  episodes  established  to  promote  action-based,  interactive  and  purposeful  exchanges  enhance  the 
potential  for  learning  content  and  culture.  The  nine  events  of  instruction:  gaining  attention , informing  learner  of  the 
objective,  stimulating  recall  of  prerequisite  learning,  presenting  the  stimulus  material,  providing  learner  guidance, 
eliciting  learner  performance,  providing  informative  feedback,  assessing  performance,  and  enhancing  retention  and  transfer 
(Gagnd,  Briggs  & Wager,  1992)  have  the  potential  to  promote  knowledge  exchanges  from  cultural  perspectives. 

Gaining  attention  can  be  accomplished  by  displaying  images  o i objects  a learner  is  likely  to  encounter  in  a 
normal  day;  and  which  relate  directly  to  content  of  the  instructional  episode.  Informing  learner  of  the  objectives  can  be 
accomplished  by  avoiding  jargon  and  using  language  common  to  the  learner  audience  without  degenerating  to  slang, 
profanity  or  colloquialism  which  may  detract  from  the  attainment  of  new  vocabulary.  Stimulating  recall  of  prerequisite 
learning  can  be  accomplished  by  asking  learners  what  is  important  about  the  intended  outcomes  of  a planned  instructional 
episode.  Presenting  the  stimulus  material  can  be  accomplished  by  asking  learners  to  identify  ideas,  thoughts  and 
perspectives  that  are  similar  and  different  from  their  own  perspective.  Providing  learner  guidance  can  be  accomplished  by 
utilizing  examples  and  artifacts  familiar  to  the  learner.  Eliciting  learner  performance  can  be  accomplished  by  encouraging 
learning  within  a familiar  cultural  context  or  within  a cultural  context  which  one  desires  to  become  familiar.  Providing 
informative  feedback  can  be  accomplished  by  comparing  and  contrasting  learner  perceptions  of  the  content  with  the 
perceptions  of  content  specialists  who  share  common  cultural  interest  with  the  learner.  Assessing  performance  can  be 
accomplished  by  encouraging  the  learner  to  compare  his  or  her  performance  with  the  expected  performance,  and  propose 
changes  that  can  be  compared  to  teacher  recommendations.  Enhancing  retention  and  transfer  can  be  accomplished  by 
varying  the  cultural  context  where  a particular  knowledge  or  skill  can  be  demonstrated  consistent  with  measures  of 
success  identified  in  the  criteria  of  the  stated  objective(s).  While  the  sequence  of  events  which  form  an  instructional 
episode  suggests  a pedagogical  framework  where  cultural  pluralism  can  be  accomplished,  there  is  no  substitute  for  the 
personal  commitment  required  to  plan  and  successfully  implement  cultural  pluralism  into  instruction. 

A learner-centered  perspective  as  a philosophical  orientation  to  instructional  technology  is  essential  in  order  to 
incorporate  cultural  pluralism  into  instruction.  Further,  there  are  four  requisites  for  promoting  cultural  pluralism: 

♦ Facilitating  learner  achievement  is  the  fundamental  purpose. 

♦ Teachers  of  any  subject  can  promote  cultural  pluralism. 

♦ There  are  invisible  privileges  inherent  in  the  classroom. 

♦ Cultural  pluralism  can  be  effectively  incorporated  into  instruction  when  planning  to  teach. 

The  main  caveat  for  people  sincerely  interested  in  instructional  technology  frameworks  that  support  cultural 
pluralism  is  not  to  make  assumptions  about  individual  values  based  on  group  norms.  Figure  1 highlights  a relation 
between  cultural  and  individual  differences. 


Six  considerations  are  recommended  prior  to  attempting  to  incorporate  cultural  pluralism  into  instruction: 

♦ Understand  your  own  perceptions. 

♦ Offer  positive  perspectives. 

♦ Prepare  sufficiently. 

♦ Set  rules  of  respect. 

♦ Begin  with  non-threatening  issues. 

♦ Optimize  the  teachable  moment.  (Branch,  Goodwin  & Gualtieri,  1993) 

The  development  of  new  conceptual  tools  and  procedures  for  responding  to  needs  for  creating  culturally  sensitive 
instruction  is  important,  however,  in  the  meantime,  instructional  technologists  should  consider  utilizing  existing 
frameworks  for  incorporating  cultural  pluralism  into  instruction. 
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(4)  Gendered  by  Design:  Reflections  on  a Social  Construct  with  Deep  Historical  Roots 

Nancy  Nelson  Knupfer,  Ph.D. 

To  the  casual  observer  an  initial  survey  of  instructional  design  theory  and  practice  might  not  reveal  any  obvious 
pattern  in  relation  to  gender.  Instructional  designers  historically  have  taken  the  "one  size  fits  all"  approach  and  until 
recent  years  have  been  called  to  task  on  this  only  mildly.  Yet  a close  examination  of  instructional  design  and  gender, 
reveals  a clear,  consistent,  and  pervasive  relationship  that  has  deep  historical  roots,  and  that  winds  throughout  our  daily 
lives  and  perpetuates  itself  through  its  interweavings  with  society. 

These  roots  continue  to  support  instructional  design  in  its  historical  sense  by  constantly  feeding  the  old  system 
while  all  but  strangling  attempts  to  pay  serious  attention  to  gender  equity  (McCormick,  1994;  Gomick  & Moran,  1972). 
Materials  developed  for  use  in  public,  private,  and  military  schools  as  well  as  instructional  messages  delivered  to  the 
public  through  advertising,  television,  and  public  service  messages  continue  to  portray  women  and  men  in  stereotypical 
ways.  Despite  attempts  to  correct  this  situation  over  many  years,  a recent  study  of  computer  clip-art  images  reveals  that 
the  stereotypes  have  invaded  the  desktop  computing  environment,  with  images  of  men  depicted  in  leadership  and 
authority  roles,  while  women  are  depicted  in  subordinate  roles  (Binns  & Branch,  1995). 

The  complexities  of  this  relationship  are  enormous,  yet  can  be  difficult  to  recognize,  reveal,  analyze,  explain, 
and  redirect.  Like  society  itself  the  complexities  reflect  the  dynamics  of  different  situations  in  different  ways,  among 
different  individuals.  An  examination  of  the  complex  relationship  between  instructional  design  and  gender  reveals 
inequities  that  result  from  a persistent  pattern  of  practice.  Recognizing  the  result  of  those  inequities  can  be  easier  than 
finding  the  causes  and  correcting  the  problem  at  its  root.  Like  a cancer,  its  result  shows  up  at  different  stages  and  in 
different  ways,  among  different  individuals.  If  left  untreated,  it  devastates  the  person.  But  recognizing  the  disease  is 
much  easier  than  finding  a reliable  cure. 

Inequities  that  result  from  the  practice  of  instructional  design  often  go  unrecognized  because  they  emerge  not 
just  as  a result  of  what  has  been  done,  but  also  as  a result  of  what  has  been  left  undone.  The  neglect  and  omission  of  the 
female  population  reveal  themselves  in  subtle  ways  on  an  individual  basis,  but  as  a collective  result  appear  throughout 
society  as  something  that  begins  in  the  home,  and  perpetuates  itself  through  schooling  and  employment  practices.  If  that 
was  not  the  case,  then  there  would  be  no  need  for  recent  efforts  to  attract  girls  into  the  study  of  math  and  science  (Kable 
& Meece,  1994)  and  the  number  of  distressing  stories  about  females  succeeding  despite  the  myriad  of  obstacles 
(Aisenberg  & Harrington,  1988;  Clark  & Corcoran,  1986;  Frenkel,  1990;  Gomick,  1990)  would  no  longer  be  told. 

Although  many  scholars  insist  that  great  strides  have  been  made  concerning  equity,  they  would  be  hard  pressed 
to  explain  why  it  was  necessary  and  yet  took  so  many  years  to  begin  such  a conversation  among  leaders  in  the  field.  Yet 
the  first  open  conversation  held  at  the  Annual  Meeting  of  Professors  of  Instructional  Design  and  Technology  concerning 
the  need  to  attend  to  equitable  treatment  of  females  and  people  of  differing  cultural  backgrounds  was  held  among  the 
delegates  in  May,  1995.  Even  sadder  is  the  fact  that  at  least  two  of  the  well-established  men  who  seemed  to  make  sense 
of  and  contributed  positively  to  the  group  conversation,  made  extremely  prejudicial  statements  following  the 
conversation.  This,  I can  only  interpret  as  either  unwillingness  or  inability  to  see  beyond  their  current  practices,  beliefs, 
and  biases.  The  real  concern  goes  beyond  the  biases  of  two  leaders,  but  to  the  larger  dimension  of  the  students  they 
teach,  the  instructional  designers  and  teachers  they  train. 

The  socially  constructed  meaning,  expectations,  and  opportunities  based  on  gender  begin  with  differing 
expectations  for  people,  depending  upon  their  sex  at  birth  (Stem  & Karraker,  1989).  The  recent  news  media  attention  to 
the  pathetic  and  heart-wrenching  plight  of  unwanted  baby  girls  who  have  been  killed  or  dumped  in  orphanages  (Chen, 
1996;  Human  Rights  Watch,  1996;  Tyler,  1996)  is  not  a value  that  is  restricted  to  Chinese  culture.  It  is  a value  the 
reflects  itself  in  the  daily  lives  and  education  of  many  people  in  many  cultures.  It  reveals  itself  in  the  way  we  groom 
boys  for  leadership  positions  while  we  teach  girls  to  be  submissive,  in  the  way  we  emphasize  tht  importance  of  male- 
dominated  sports,  in  the  way  teachers  respond  to  boys  differently  than  to  girls  (Olivares  & Rosenthal,  1992),  in  the  way 
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stereotypes  are  perpetuated  in  the  media  (Kilboume,  1990;  Schwartz  & Markham,  1985),  and  in  the  way  we  recruit  for 
jobs  (Bern  & Bern,  1973;  Fidell,  1975;  Rowe,  1990).  It  reveals  itself  in  the  way  we  provide  examples,  exercises,  and 
meaningful  educational  opportunities  that  boys  can  often  relate  to  better  than  girls.  It  reveals  itself  in  the  grooming  of 
boys  for  entire  categories  of  jobs  involving  science,  math,  medicine,  and  politics.  For  example,  can  you  explain  how  a 
girl  would  know  that  national  television  advertisements  about  hotline  counseling  at  "Boystown"  possibly  could  be  meant 
to  serve  girls  as  well  as  boys?  Never  a mention  of  this  is  made.  Or  perhaps  you  could  explain  why  there  is  a dearth  of 
girls  applying  to  attend  ' Boys  State"  during  special  summer  sessions  at  college  campuses.  Does  our  society  really  wish 
to  imply  that  gender  is  so  very  important  in  shaping  future  political  leaders  that  the  very  name  of  a program  must 
discourage  girls  from  attending? 

Girls  can  achieve  equally  well  in  the  aforementioned  areas  but  have  not  been  encouraged  to  do  so  until  recently. 
And  now  the  attempts  are  filled  with  remaining  hurtles  and  barriers  that  must  be  overcome  (Top,  1991).  Meaningful 
instructional  design  practice  must  do  more  to  attend  to  these  matters  and  take  an  active  role  in  encouraging  girls  (Van 
Nostrand,  1991).  While  not  enough  has  been  done  and  it  is  too  late  for  many,  instructional  designers  can  begin  to  make 
amends  for  those  girls  who  have  yet  to  come  through  our  nation's  school  systems  and  workforce  training  programs. 
Instructional  designers  can  make  a better  effort  to  provide  experiences  that  girls  can  relate  to,  offering  instructional 
opportunities  that  are  not  gender  biased,  and  encouraging  teachers  to  actively  attend  to  issues  of  gender  equity  (Turkle  & 
Papert  1990). 

Instructional  designers  can  influence  the  entertainment  industry,  home  market,  school  environment,  and  practices 
in  business  and  military  environments.  Designers  can  accept  the  importance  of  their  role  in  shaping  the  self  concept  and 
encouraging  equitable  access  to  job  skills  that  later  translate  into  life  skills  and  wages  commensurate  with  experience.  A 
more  positive  and  forward-thinking  outlook  on  the  role  of  females  in  our  society  can  certainly  do  much  to  influence  the 
drive  and  effort  that  is  currently  necessary  for  females  to  overcome  the  many  obstacles  in  daily  life. 

Instructional  designers  can  influence  educational  practice  by  designing  instructional  environments  that  attend  to 
the  needs  of  the  female  population  as  well  as  those  of  males.  They  can  encourage  reflective  practice  that  makes 
adjustments  to  the  needs  at  hand.  The  information  age  brings  the  challenge  of  shifting  responsibilities.  Even  through 
the  majority  of  network  users  are  males  (Shade,  1993),  females  must  be  encouraged  to  learn  skills  and  be  provided  with 
opportunities  to  have  equal  access  to  information. 

Until  society  gets  beyond  viewing  women  as  second  class  citizens  in  stereotypical  roles,  then  instructional 
designers  will  have  a difficult  job  in  educating  the  public.  Yet  it  can  be  done.  The  first  step  is  to  educate  instructional 
designers  to  attend  to  the  needs  of  a pluralistic  society.  The  second  step  is  to  encourage  business,  industry,  government, 
and  education  institutions  to  include  knowledgeable  designers  on  their  project  development  teams.  The  designers  can 
produce  text-based  and  mediated  materials  that  attend  to  the  needs  of  females  as  well  as  males.  Further,  they  can  provide 
better  training  for  teachers  and  others  who  provide  information  to  the  masses. 

4 

References 

Aisenberg,  N.  & Harrington,  M.  (1988).  Women  in  academe:  Outsiders  in  the  sacred  grove.  Amherst,  MA:  University 
of  Massachusetts  Press. 

Bern,  S.  L.  & Bern,  D.  J.  (1973).  Does  Sex-biased  Job  Advertising  "Aid  and  Abet*  Sex  Discrimination?  Journal  of 
Applied  Social  Psychology  3(1),  pp.  6-18. 

Binns,  J.  C.  & Branch,  R.  C.  (1995).  Gender  stereotyped  computer  clip-art  images  as  an  implicit  influence  in 

instructional  message  design.  In  D.  G.  Beauchamp,  R.  A.  Braden,  & R.  E.  Griffin  (Eds.),  Imagery  and  visual 
literacy  (pp.  315-324).  Rochester,  NY:  International  Visual  Literacy  Association. 

Chen,  Y.  (1996,  Feb.)  Personal  interview  with  visiting  scholar  about  education,  illiteracy,  and  gender.  Conducted  by  N. 
N.  Knupfet  at  Kansas  State  University,  Manhattan,  KS. 

Clark,  S.  M.  & Corcoran,  M.  (1986,  Jan./Feb.).  Perspectives  on  the  professional  socialization  of  women  faculty:  A 
case  of  Accumulative  Disadvantage?.  Journal  of  Higher  Education,  57(1 ). 

Couch,  R.  A.  (1995).  Gender  equity  & visual  literacy:  Schools  can  help  change  perceptions.  In  D.  G.  3eauchamp,  R. 
A.  Braden,  & R.  E.  Griffin  (Eds.),  Imagery  and  visual  literacy  (pp.  105-111).  Rochester,  NY:  International 
Visual  Literacy  Association. 

Fidell,  L.  S.  (1975).  Empirical  verification  of  sex  discrimination  in  hiring  practices  in  psychology,  in  R.  K.  Unger  & 

F.  L.  Denmark  (Eds.)  Women:  Dependent  or  independent  variable.  New  York,  NY:  Psychological  Dimensions. 
Frenkel,  K.  A.  (1990,  Nov.).  Women  and  computing.  Communications  of  the  ACM,  pp.  34-46. 

Gomick,  V.  (1990).  Women  in  science:  100  Journeys  into  the  territory.  New  York,  NY:  Touchstone,  a.Division  of 
Simon  & Schuster). 


554 


IT 

c 


;gs 


Gornick,  V.  & Moran,  B.  K.  (Eds.)  (1972).  Women  in  sexist  society.  New  York,  NY:  Basic  Books. 

Human  Rights  Watch/Asia  (1996).  1/96  Death  by  default  - summary  and  recommendations.  New  York,  NY:  Human 
Rights  Watch. 

Kable,  J.  B.  & Meece,  J.  (1994).  Research  on  gender  issues  in  the  classroom.  In  D.  L.  Gabel  (Ed.),  Handbook  of 
research  on  science  teaching  and  learning  (542-557).  New  York,  NY:  Macmillan  Publishing  Co. 

Kilboume,  W.  E.  (1990).  Female  stereotyping  in  advertising:  An  experiment  on  male-female  perceptions  of  leadership. 
Journalism  Quarterly  67(1),  25-31. 

McCormick,  T.  M.  (1994).  Creating  the  nonsexist  classroom:  A multicultural  approach.  New  York,  NY:  Teachers 
College  Press. 

Olivares,  R.  A.  & Rosenthal,  N.  (1992).  Gender  equity  and  classroom  experiences:  A review  of  research.  ERIC 
document  #ED366701 . 

Rowe,  M.  P.  (1990).  Barriers  to  equality:  The  power  of  subtle  discrimination  to  maintain  unequal  opportunity. 
Employee  Responsibilities  and  Rights  Journal  3(2),  pp.  153  - 163. 

Schwartz,  L.  A.  and  W.  T.  Markham  (1985).  Sex  stereotyping  in  children's  toy  advertisements.  Sex  Roles:  A Journal 
of  Research,  12,  pp.  157-170. 

Shade,  L.  R.  (1993).  Gender  issues  in  computer  networking.  Talk  given  at  Community  Networking:  the  International 
Free-Net  Conference,  August  17-19.  Carleton  University,  Ottawa,  Canada. 

Stern,  M.  & K.  H.  Karraker  (1989).  Sex  stereotyping  of  infants:  A review  of  gender  labeling  studies.  Sex  Roles:  A 
Journal  of  Research,  20(1),  pp.  501-522. 

Top,  T.  J.  (1991).  Sex  bias  in  the  evaluation  of  performance  in  the  scientific,  artistic,  and  literary  professions:  A 
review".  Sex  Roles:  A Journal  of  Research,  24,  pp.  73  - 106. 

Turkle,  S.  & Papert,  S.  (1990).  Epistemological  pluralism:  Styles  and  voices  within  the  computer  culture".  Signs: 
Journal  of  Women  in  Culture  and  Society,  16,  pp.  128  - 157. 

Tyler,  P.  E.(  1996).  In  China's  orphanages,  a war  of  perception.  In  The  New  York  Times  newspaper,  Sunday,  January 
21,  pp.  H-31. 

Van  Nostrand,  C.  H.  (1991).  Gender-Responsible  Leadership:  Do  Your  Teaching  Methods  Empower  Women?  NY: 
Sage  Publications,  Inc. 


os 

(5)  The  Need  for  Explicit  Instructional  Design  Paradigms 
Wendy  Thompson 

Instructional  design  responds  to  the  complexities  of  instruction  in  numerous,  unique  ways.  Though  the  goal  of 
each  domain  of  instructional  design  is  to  maximize  the  instructional  intervention,  however,  paradigms  that  conceptualize 
learner-centered  instruction  have  yet  to  address  culture  and  its  importance  in  the  instructional  process. 

Education  is  a social  system.  Like  other  social  systems,  it  mirrors  attitudinal  change  over  time.  Qualitative 
research's  challenge  to  the  preeminence  of  quantitative  research  in  academic  literature  (see  Bogdan  & Biklen,  1992)  and  the 
shift  from  chalkboards  to  computers  in  the  classroom  illustrate  this  assertion. 

America,  once  perceived  as  "The  Great  Melting  Pot,"  preached  Americanization  by  assimilation.  The  attempt 
was  made  to  strip  away  or  minimize  ethnicity;  an  amalgam  of  people  was  to  make  one  America.  The  relatively  new 
immigration  and  a surge  in  cultural  pride,  however,  challenged  America's  historical  assimilationist  paradigm,  fostering 
the  perception  of  this  country  as  a mosaic.  Each  tile  with  its  own  shape,  size,  and  color  made  a contribution  to  the 
beauty  of  the  overall  picture.  This,  according  to  Scheel  and  Branch  (1993)  constitutes  cultural  pluralism,  a concept 
which  "reflects  the  recognition  that  cultures  have  no  inherent  hierarchy  of  truthfulness;  that  is,  patterns  of  behavior  and 
thinking  yield  a multiplicity  of  perceptions  of  the  world  which  are  no  more  or  less  verifiable  than  others"  (p.7).  Cultural 
pluralism  may  be  seen  as  the  ultimate  goal  evolving  from  a diverse  population. 

While  the  country  - and  the  world  for  that  matter  - had  undergone  a major  paradigmatic  shift  on  a macro  social 
level,  a shift  of  this  magnitude  had  yet  to  take  place  on  a micro  level  within  education,  a field  which  has  been 
traditionally  assimilationist.  The  culture  of  those  involved  in  the  educational  process  was  not  a determinant  factor, 
because  the  primary  goal  of  education  was  the  mainstreaming  of  all  groups.  The  assimilationist  educator,  then,  may 
have  advocated  English  as  a second  language  instruction  not  for  the  enrichment  of  the  students,  but  as  a quick-fix  to  get 
immigrants  to  become  literate  English  speakers  (Janzen,  1994).  The  acceptance  of  the  tenets  of  instructional  design  over 
the  years  has  gradually  minimized  instances  of  outmoded,  teacher-centered  instruction. 
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While  instructional  design  itself  is  being  accepted  more  readily,  educators  appear  to  be  reticent  in  incorporating 
the:  cultural  renaissance  into  instructional  planning.  Instructional  design  paradigms  remain  stagnant  in  the  face  of 
sweeping  cultural  reform.  The  culture  of  all  participants  within  the  instructional  intervention  should  be  made  more 
explicit.  Models  are  needed  to  reflect  culture's  presence. 

Presently,  educational  technologists  are  functioning  on  a type  of  honor  system.  One  is  left  to  assume  that 
cultural  elements  are  included  in  instruction.  Banks’  Four  Levels  of  Implementation  (1989)  describes  the  depth  of 
inclusion  of  culture  in  curricula.  Level  I,  Contributions,  acknowledges  non-mainstream  contributions,  highlighting 
individually  distinct  elements,  such  as  heroes,  holidays,  and  foods.  Level  II,  Additive,  allows  for  the  insertion  of  content, 
themes,  and  perspectives  of  cultural  groups  into  the  curriculum;  basic  curricular  structure  remains  unchanged.  At  Level 
IB,  Transformation,  structural  changes  to  the  curriculum  allow  for  the  inclusion  of  concepts,  issues,  and  events  viewed 
from  the  perspective  of  the  cultural  group.  Level  IV,  Social  Action,  appears  to  be  the  most  self-actualized  level  of  the 
paradigm.  Students  decide  on  social  issues  involving  cultural  groups  and  act  constructively  to  solve  them.  Presently, 
most  curricular  inclusion  of  culture  occurs  at  Level  I.  Quaint  observations  of  holidays,  sampling  of  different  foods,  and 
exhibitions  of  native  dress  suffice  for  the  inclusion  of  culture  in  curricula.  Instructional  design  models  currently  in  use 
accommodate  this  implicit  inclusion. 

Instructional  designers  should  be  mindful  of  the  position  that  learning  occurs  like  Lego®  blocks,  building  on 
itself  to  form  new  ideas.  The  more  attractive  the  block,  the  greater  the  chance  that  it  will  be  snapped  from  short  term 
memory  snugly  into  place  in  long  term  memory,  to  last  ad  infinitum.  Chunking  and  the  acceptability  of  messages 
increase  exponentially  the  opportunities  for  these  blocks  to  snap  together  and  last,  ensuring  that  learning  has  occurred 
(Gagn6,  Briggs,  & Wager,  1992;  Gredler,  1992).  Common  sense  dictates  that  messages  coming  from  a messenger  that 
most  resembles  the  learner  would  make  the  message  more  attractive  and  would  hasten  its  adherence  to  the  learner’s 
schema.  This  principle  is  one  of  the  basic  foundations  of  culturally  pluralistic  instruction. 

One  may  ask,  "What’s  the  solution,  then,  to  including  culture  in  instructional  design?"  Is  in  tweaking  models 
such  as  Dick  and  Carey  by  merely  adding  another  box?  Is  it  in  reshuffling  and  adding  another  event  to  Gagnes  already 
existing  nine?  Or  is  it  in  totally  discounting  present  instructional  design  models  altogether?  General  system  theory  is 
one  of  the  informants  of  instructional  design  (see  Dick  & Carey,  1996);  basic  systems  theory  states  that  entropy  is 
inherent  in  all  systems.  Models  of  the  past  have  nobly  served  their  purpose  in  the  systems  for  which  they  had  been 
designed.  However,  those  systems  are  breaking  down,  making  way  for  a new  culturally  pluralistic  world  order.  The  time 
has  come  for  new  instructional  design  models  that  conceptualize  and  reflect  this  change.  A new  generation  of 
instructional  designers  appears  ready  to  rise  to  the  challenge  of  their  creation. 
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(6)  An  Evaluator  Considers  the  Cultural  Sensitivity  of  Instructional  Materials 
Thomas  C.  Reeves,  Ph.D. 

Is  cultural  sensitivity  a serious  issue  in  the  design,  dissemination,  and  evaluation  of  instructional  programs  and 
products  for  international  markets?  Some  might  argue  that  cultural  sensitivity  is  just  a passing  "fad,"  a by-product  of  the 
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current  attention  to  multiculturalism  within  North  American  and  European  academic  institutions.  I disagree.  Sensitivity 
to  cultural  diversity  and  pluralism  is  a "meta- value"  that  should  influence  virtually  every  aspect  of  human  activity, 
including  instructional  systems  design. 

The  rationale  for  including  cultural  diversity  as  a critical  factor  in  evaluating  instructional  programs  and  products 
goes  to  the  heart  of  the  challenge  of  making  evaluation  a legal  and  ethical  process.  Scriven  (1993)  describes  ethical, 
legal,  and  ecological  standards  as  "value  absolutes"  that  must  not  be  violated  when  evaluating  the  merit  and  value  of 
programs  and  products.  For  example,  practical  criteria  for  evaluating  computer-based  training  (CBT)  may  include  costs, 
but  an  ethical  absolute  would  be  paramount  if  it  was  found  that  the  CBT  delivery  system  released  cancer-causing 
emissions.  Cultural  sensitivity  should  be  added  to  the  list  of  value  absolutes.  Attention  to  issues  involved  in  cultural 
and  ethnic  diversity  are  not  "nice  to  have"  criteria,  but  essential  elements  in  the  evaluation  of  education  and  training.  The 
importance  of  this  absolute  is  heightened  in  the  1990's  as  more  and  more  commercial  and  academic  institutions  seek  to 
market  instructional  systems  around  the  globe. 

Any  instructional  designers  and  evaluators  who  have  worked  internationally  or  with  minority  populations  in 
their  own  countries  can  identify  examples  of  instructional  programs,  products,  or  methods  that  are  culturally  insensitive. 
For  instance,  when  I was  in  Taiwan  several  years  ago,  I met  the  president  of  a large  electronics  company  who  was 
interested  in  providing  ESL  (English  as  a Second  Language)  instruction  for  his  employees  and  their  children.  This 
entrepreneur  was  convinced  that  computer-based  instruction  could  play  a significant  role  in  this  undertaking,  but  he  was 
disgusted  with  the  cultural  bias  of  commercial  ESL  materials  he  had  purchased  from  the  USA  and  the  UK.  "These 
programs  ignore  the  importance  of  the  family  in  Chinese  culture,"  he  complained.  As  an  illustration,  he  showed  me  a 
writing  assignment  from  one  program.  Students  were  supposed  to  complete  a paragraph  that  began:  "Johnny  is  worried 
about  going  home  today  because  his  mothers  boyfriend  is  coming  to  dinner  for  the  first  time.  Johnny  feels...."  The 
insensitivity  of  this  exercise  to  Chinese  cultural  norms  needs  little  explanation. 

Ironically,  the  designers  of  this  ESL  exercise  may  have  been  trying  to  be  sensitive  to  learners  who  came  from 
broken  homes,  consciously  deciding  to  place  Johnny  in  a dilemma  which  is  not  unfamiliar  in  some  Western  societies. 
While  this  exercise  might  have  been  appropriate  for  some  cultures,  marketing  these  materials  to  another  culture  without 
considering  the  appropriateness  of  this  scenario  in  that  other  culture  is  an  insensitive  act. 

Cultural  sensitivity  is  often  a subtle  issue,  and  it  is  very  naive  of  instructional  designers  and  evaluators  to 
imagine  that  they  can  detect  cultural  insensitivities  on  their  own.  While  some  checklists  or  rating  scales  may  assist 
designers  and  evaluators  in  catching  glaring  examples  of  cultural  insensitivity,  the  collaboration  of  representatives  of  each 
target  culture  is  necessary  to  identify  less  obvious  sources  of  cultural  bias.  Andrews  (1994)  provides  numerous  examples 
of  the  hidden  nature  of  cultural  bias  within  a South  African  context.  For  example,  Andrews'  paper  includes  a graphic  of  a 
boy  kissing  his  mother  from  a computer-based  language  lesson.  The  picture  had  to  be  changed  to  that  of  a mother 
kissing  a boy  when  a Zulu-speaking  member  of  the  development  team  indicated  that  it  is  inappropriate  for  a boy  to  kiss 
his  mother  in  the  Zulu  culture.  Andrews  also  points  to  the  common  use  of  heads  and  hands  as  icons  in  multimedia; 
these  are  seen  as  taboo  images  of  severed  body  parts  in  some  African  cultures.  Leh  (1996)  illustrates  other  examples  of 
the  cultural  insensitivity  of  multimedia  design  factors  from  an  Islamic  perspective.  For  example,  the  common  use  of 
animals  as  metaphors  for  people  in  "edutainment"  software  for  children  can  be  highly  insulting  to  Muslims,  especially 
animals  such  as  dogs,  pigs,  and  rats. 

Even  core  pedagogical  values  in  one  culture  may  be  culturally  insensitive  in  another.  An  "advanced"  science 
education  curriculum  for  primary  children  developed  by  the  ministry  of  education  in  an  Australian  state  stresses  the 
asking  of  "Why?"  questions.  This  same  curriculum  contradicted  cultural  traditions  within  some  Aboriginal  communities 
in  which  asking  "Why?"  to  an  elder  such  as  a teacher  is  considered  highly  insolent.  The  solution  may  not  be  in 
eliminating  "Why?"  questions  from  the  science  curriculum,  but  extra  attention  to  how  innovative  pedagogical  strategies 
are  introduced  into  minority  cultures  is  required. 

There  is  not  room  in  this  brief  presentation  to  describe  all  the  ways  that  evaluators  and  others  can  include 
cultural  sensitivity  as  a meta-value  in  their  evaluations  of  instructional  materials.  Further,  we  hardly  know  enough  about 
this  important  issue  to  delineate  all  the  ramifications  of  this  process.  In  fact,  this  is  an  area  in  much  need  of  creativity 
and  research.  At  the  same  time,  it  is  important  to  point  out  that  cultural  sensitivity  is  not  just  a matter  of  identifying 
and  eliminating  cultural  bias.  The  culturally  sensitive  instructional  designer  and  evaluator  should  be  proactive  by  seeking 
opportunities  to  increase  the  cultural  relevance  of  instructional  materials  and  to  build  upon  cultural  diversity  and 
pluralism.  Our  ultimate  goal  should  not  be  designing  culturally  neutral  instructional  materials,  but  creating  learning 
environments  that  are  enriched  by  the  unique  values  that  are  inherent  in  different  cultures.  This  is  much  easier  said  than 
done,  but  it  is  one  of  the  most  important  challenges  faced  by  educational  technologists  as  we  approach  the  2 1st  Century. 
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Abstract 


This  study  examined  the  role  taking  notes  using  either  pencil  and  paper  and  an  on-line  computer  notepad  during 
computer-delivered  instruction  have  on  retention  and  depth  of  processing.  The  results  of  this  study  can  be 
summarized  as  follows:  1.  There  was  a difference  in  total  post-test  scores  between  the  control  group  which 
took  no  notes  and  the  treatment  group  which  took  notes  on-line.  The  difference,  which  supported  on-line 
notetaking,  appeared  to  be  due  to  the  way  the  control  and  on-line  notetakers  answered  the  factual-type  questions 
on  the  post-test;  2.  The  type  of  notes  taken  did  not  affect  the  overall  retention  of  information;  3.  The  method 
used  to  take  notes  and  the  type  of  notes  taken  did  not  affect  the  number  of  idea  units  recorded  by  the  subjects;  4. 
A difference  in  recall  scores  was  found  between  the  verbatim  and  both  the  own  style  and  paraphrase  notetakers. 

It  supported  the  verbatim  notetakers;  5.  There  was  a positive  correlation  between  the  recall  scores  and  total 
post-test  scores  of  the  subjects. 


Introduction 


The  perception  by  both  educators  and  students  that  notetaking  is  a powerful  strategy  which  enhances  recall  and 
learning  from  lecture  and  text  is  supported  by  research.  To  date,  most  notetaking  research  has  centered  around  the 
encoding  and  external  storage  functions  of  notetaking  as  first  proposed  by  Di  Vesta  and  Gray  (1972).  The  encoding 
hypothesis  suggests  that  notetaking  serves  to  increase  the  learner's  attention  during  lecture  or  when  reading  from  text, 
thereby  encouraging  learners  to  integrate  and  elaborate  upon  what  they  hear,  see,  or  read  with  their  prior  knowledge. 

This,  in  turn,  increases  the  chances  of  encoding  the  noted  information  in  long-term  memory. 

The  external  storage  function,  on  the  other  hand,  asserts  that  notes  themselves  are  beneficial  only  when  used  as  a 
review  tool.  In  other  words,  the  process  of  taking  notes  is  not  important  in  recall  and  retention.  It  is  only  the  product, 
the  notes  themselves,  when  employed  for  review,  that  are  important. 

In  reality,  students  most  likely  take  notes  because  of  both  the  encoding  and  storage  functions  of  notetaking 
(Hartley  & Davies,  1978).  In  other  words,  students  take  notes  to  maintain  a written  record  of  what  occurred  in  lecture  or 
text  and  later  use  this  record  to  enhance  review.  The  benefits  of  the  notetaking  plus  review  strategy  has  been  well 
documented.  In  studies  where  notetaking-only  groups  have  been  compared  to  notetaking-plus-review  groups  and  no-notes 
groups,  in  general,  the  notetaking-plus-review  groups  yielded  superior  recall  (Fisher  & Harris,  1973;  Carter  & Van 
Matre,1975;  Rickards  & Friedman,  1978).  In  a review  of  26  notetaking  comparison  studies,  Ganske  (1981)  consistently 
found  that  students  who  took  notes  outperformed  those  who  did  not  take  notes  on  recall  tests  when  a period  of  time  was 
provided  to  review  notes.  Instead  of  trying  to  isolate  whether  the  encoding  or  storage  function  is  more  significant,  it  may 
be  of  more  importance  to  focus  on  the  type  of  information  included  in  notes  which  may  be  indicative  of  the  level  of 
processing  used  to  produce  the  notes  as  well  as  their  utility  as  a review  tool. 

The  movement  from  behaviorism  to  cognitivism  has  significantly  influenced  the  design  of  instruction  delivered 
via  computers.  The  emphasis  has  moved  from  using  the  computer  to  deliver  simple  page  turn  : software  to  software 
which  encourages  learners  to  actively  participate  in  instruction  through  lesson  sequencing  options,  embedded  questions 
and  answers,  predicting  and  hypothesizing,  and  customized,  specific  feedback  (Hannafin  & Reiber,  1987;  Yankelovich  & 
van  Dam,  1985;  Grubaugh,  1985;  Neuwrith,  C.  M.  Kaufer,  D.,  1987). 

Today  notetaking  applications  available  on  microcomputers  range  on  a continuum  from  utility-type 
applications,  such  as  calendars  or  note-posting  options,  to  sophisticated  hypertextual  applications  used  to  facilitate 
recording,  organizing  and  manipulation  of  the  notes. 

There  are  several  similarities  between  computer  and  text  based  learning  environments.  Most  computer-delivered 
instruction  and  prose  offer  learners  the  ability  to: 

• move  to  previous  prose  for  additional  review  and  clarification; 

• reread  passages  until  they  are  understood; 

• attend  to  information  at  a pace  determined  by  the  learner; 

• use  cues  such  as  layouts,  formats  or  heading  embedded  in  the  text  as  visual  aids  to  help  determine  important 
information. 

Lecture  environments  on  the  other  hand,  require  learners  to  follow  at  a pace  determined  by  the  instructor  which, 
according  to  Peters  (1972),  may  be  at  a rate  up  to  180  words  per  minute.  If  visua'  aids  are  not  used,  students  in  lecture 
environments  must  often  rely  only  upon  verbal  inflections  for  cueing.  The  similarities  between  learning  from  prose  and 


from  computer-delivered  instruction  suggest  that  research  methodologies  used  to  assess  notetaking  from  text  may  be 
helpful  in  evaluating  the  role  of  notetaking  during  computer-delivered  instruction. 

Although  the  research  supporting  the  utility  of  notetaking  in  the  prose  environment,  which  is  similar  to  that  of 
the  computer-delivered  instructional  mode,  is  very  positive,  there  exists  a scant  amount  of  research  which  examines  the 
role  of  notetaking  using  either  pencil  and  paper  or  a computer  during  computer-delivered  instruction  itself. 

In  research  related  to  computers  and  notetaking,  Novell  i no  (1985)  compared  computerized  note  taking  and 
notetaking  using  pencil  and  paper  in  a lecture  environment.  Subjects  were  divided  into  four  groups  based  on  notetaking 
method  (computerized  and  pencil  and  paper)  and  gender.  Each  subject  listened  to  four  lectures  and  recorded  notes  during 
each  session  using  one  of  the  methods  assigned.  After  each  session  a post-test  was  administered  in  which  the  recall  of 
total  words,  key  words,  and  ideas  were  assessed.  The  results  indicated  that  notetaking  using  pencil  and  paper  yielded 
greater  recall  for  subjects  who  were  poor  typists  while  notetaking  via  a microcomputer  resulted  in  greater  recall  for  skilled 
typists.  Significant  differences  in  the  time  required  to  complete  the  lesson  favored  groups  using  the  microcomputer.  No 
attempt  was  made  to  examine  the  notes  themselves  to  determine  if  the  computerized  method  facilitated  greater 
assimilation,  elaboration,  or  reorganization  on  the  part  of  the  subjects. 

Only  one  study  was  located  which  examined  the  role  of  computerized  notetaking  as  an  integral  part  of  a lesson 
delivered  via  the  computer.  Wambaugh  (1991)  assigned  undergraduate  students  to  one  of  these  three  random  treatment 
conditions  during  the  computer-delivered  instruction: 

• a control  group  which  was  allowed  to  take  no  notes; 

• a group  which  created  notes  using  a computer  and  stored  them  in  a computerized  note  file; 

• a group  which  captured  notes  from  the  instruction  by  highlighting  and  copying  them  to  a computer  note  file. 

Wambaugh  reported  three  major  findings  of  this  study.  First,  learners  who  created  their  own  notes  during 

computer-delivered  instruction  experienced  a significantly  higher  level  of  achievement  on  both  immediate  and  delayed 
post-tests  than  learners  who  captured  notes.  However,  the  group  that  created  their  own  notes  did  not  outperform  the 
control  group  who  took  no  notes.  Second,  there  was  no  correlation  between  either  the  immediate  and  delayed  post-te^c s 
scores  and  the  number  of  words  in  the  computerized  note  files  created  by  the  two  groups  of  notetakers.  Third,  there  was 
no  correlation  between  achievement  on  either  the  immediate  or  delayed  post-tests  and  the  number  of  ideas  recorded  in  the 
subjects'  notes  which  were  tested. 

Wambaugh  contended  that  no  correlations  between  the  immediate  and  delayed  post-test  scores,  the  number  of 
words  in  the  note  files,  and  number  of  ideas  tested  may  be  due  to  the  type  of  achievement  test  administered.  Both  tests 
required  the  learner  to  type  responses  as  a phrase  or  few  words  rather  than  select  the  correct  answer  from  a list  of  possible 
answers.  Also,  the  computer  delivery  system  did  not  allow  learners  to  look  back  at  previous  questions  for  possib!e  clues. 

This  study  supported  the  depth  of  processing  hypothesis  which  contends  that  the  more  learners  interact  with  new 
material  in  an  attempt  to  make  it  their  own,  the  more  likely  it  is  to  be  retained  in  memory.  This  was  accomplished 
during  the  study  by  allowing  learners  to  create  their  own  notes.  Unfortunately,  Wambaugh  did  not  examine  the  notes  to 
determine  the  degree  or  depth  to  which  students  did  process  the  information. 

Three  observations  related  to  notetaking  research  in  general  influenced  the  design  of  the  present  study.  First,  the 
duration  of  most  reported  notetaking  studies  has  been  very  limited.  Research  studies  involving  notetaking  during  lecture 
were  often  limited  to  a 50-minute  class  period  or  less,  while  research  ffom  prose  was  often  restricted  to  taking  notes  from 
passages  that  were  less  than  2,000  words  in  length.  It  is  questionable  whether  such  research  settings  represent  realistic 
notetaking  situations  (Howe,  1974;  Hartely  & Marshall,  1974). 

Second,  although  Wittrock  (1974,  1978)  has  suggested  that  benefits  are  gained  from  notetaking  when  learners 
generate  paraphrased  notes  which  incorporate  prior  knowledge,  few  research  studies  have  attempted  to  examine  the  notes 
themselves  to  determine  if  generative  notes  are  in  fact  routinely  taken  by  learners.  Most  research  simply  assumes  that 
when  instructed  to  take  paraphrased  notes,  subjects  produce  notes  which  reflect  a deep  level  of  processing  (Ganske,  1984; 
Kiewra,  1987). 

Finally,  most  research  related  to  notetaking  has  been  done  utilizing  lectures  and  prose.  Although  several 
researchers  have  indicated  the  potential  benefit  of  notetaking  during  computer-delivered  instruction  (Hannafin,  1989; 
Kozma,  1987;  Jonassen,  1988),  little  research  has  been  undertaken  to  assess  its  merits  or  to  evaluate  the  use  of  new 
procedures  available  to  take  notes  on-line. 

The  current  study  attempted  to  expand  for  students,  educators,  and  software  designers,  the  limited  research  base 
related  to  computer-delivered  instruction  and  notetaking  by  addressing  the  aforementioned  research  issues  through: 

• involving  the  subjects  in  notetaking  during  computer-delivered  instruction  weekly  for  a period  of  four  weeks; 

• using  a computerized  notepad  as  well  as  pencil  and  paper  as  mediums  to  record  notes; 
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assessing  the  generative  process  of  notetaking  by  evaluating  weekly  the  content  of  the  three  types  of  notes 
(own  style,  verbatim,  and  paraphrase)  recorded  by  subjects  using  pencil  and  paper  and  an  on-line  notepad. 

Procedures 


Subjects 

The  sample  Qi- M2)  consisted  of  junior  and  senior  undergraduates  from  a southern  Minnesota  university  whose 
major  was  computer  and  information  science.  Subjects  with  this  major  were  selected  to  participate  in  the  study  because 
their  discipline  required  significant  time  using  computer  technology.  This  also  assured  that  all  subjects  had  adequate 
experience  with  computers  prior  to  the  study,  thus  minimizing  the  potential  novelty  effect  often  associated  with 
computer-related  research. 

Students  who  participated  in  this  study  were  enrolled  in  a computer  and  information  science  course  during  the 
study  period.  Participation  in  the  study  was  part  of  their  assigned  course  work  for  the  quarter.  Their  performance  on  the 
post-test  measure  did  not  impact  the  grade  they  received  in  the  course. 

All  students  completed  a questionnaire  prior  to  the  start  of  the  study.  Students  who  indicated  they  had  no  prior 
knowledge  of  the  technical  aspects  of  IBM’s  AS/400  minicomputer  system  and  had  worked  extensively  with 
microcomputers  and  word  processing  applications  were  eligible  for  the  study.  Based  on  these  criteria,  21  subjects  from 
the  original  pool  of  133  were  omitted  from  the  study  leaving  1 12  final  participants. 

Materials 

Tutorial.  This  study  required  participants  to  view  modules  from  the  AS/400  tutorial  and  take  notes  from  the 
modules  using  either  pencil  and  paper  or  an  on-line  computer  notepad.  These  modules  were  selected:  Operating  System 
and  Architecture  Support,  Equipment  Overview,  Control  Language  Structure,  Attachment  of  Personal  Computers  and 
Other  Devices.  Based  on  the  observations  conducted  using  a similar  student  sample  to  view  these  modules,  it  was 
estimated  that  each  module  would  take  approximately  40-60  minutes  to  view. 

A week  after  the  subjects  finished  the  modules,  each  participant  completed  an  exit  questionnaire  and  took  a post- 
test. 


Idea  units.  Idea  units  contained  in  each  of  the  modules  viewed  by  the  subjects  were  determined.  An  idea  un' 
was  defined  as  a single  complete  idea  or  single  block  of  information  (Bransford  & Johnson,  1972).  A total  of  32  idea 
units  from  the  four  modules  were  selected.  These  idea  units  served  as  the  basis  for  assessing  notes. 

Post-test.  The  achievement  post-test  consisted  of  40  questions:  22  multiple-choice,  13  fill-in-the  blank,  and 
four  true/false.  The  questions  included  32  factual  and  eight  synthesis-type  questions.  The  test  was  completed  using  paper 
and  pencil.  All  groups,  except  the  control  group  who  viewed  the  modules  and  took  no  notes,  were  allowed  to  review 
their  notes  prior  to  the  test.  The  control  group  was  allowed  to  mentally  review  the  content  of  the  modules.  The 
reliability  of  the  post-test  instrument  was  .78  as  determined  by  a KR-20  statistical  procedure. 

Exit  questionnaire  . This  served  as  a tool  to  elicit  the  amount  of  time  subjects  spent  reviewing  prior  to  the  post- 
test and  their  preference  for  text-based  or  computer-delivered  instructional  format 

Treatment 

Subject  groups.  The  subjects  participating  in  the  study  were  randomly  assigned  to  groups  which  served  to 
assess  the  benefits  of  both  the  notetaking  method  (pencil  and  paper,  on-line  notepad,  or  no  notes)  and  the  type  of  notes 
taken  (own  style,  verbatim,  or  paraphrased). 

Those  subjects  who  were  assigned  last  digits  from  one  to  seven  participated  in  the  study  and  their  data  was  used 
in  the  analysis  of  the  research.  Those  subjects  who  were  eliminated  from  the  study  due  to  responses  on  the  pre-study 
questionnaire  were  assigned  a last  digit  of  eight.  This  group  completed  the  modules  and  took  the  post-test,  but  their  data 
was  not  used  in  any  phase  of  the  analysis. 

The  following  list  identifies  the  significance  of  each  of  the  eight  digits  used  to  identify  the  group  assignment  of 
each  study  participant: 

1 - served  as  the  control  group,  only  viewed  the  modules  and  took  no  notes; 

2- took  notes  using  their  own  notetaking  style  with  pencil  and  paper; 
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3-  took  verbatim  notes  with  pencil  and  paper; 

4- took  paraphrased  notes  with  pencil  and  paper; 

5-  took  notes  using  their  own  notetaking  style  with  the  on-line  notepad; 

6- - took  verbatim  notes  with  the  on-line  notepad; 

7- took  paraphrased  notes  with  the  on-line  notepad; 

8- data  not  used  in  the  analysis,  took  notes  from  modules  two  and  four  using  any  method  and  type  of  notes. 

For  purposes  of  data  analysis,  each  subject  in  this  study  had  membership  in  two  groups  associated  with  the 

method  and/or  type  of  notes  taken.  First,  three  notetaking  method  groups  included  subjects  who  took  no  notes,  notes 
using  pencil  and  paper,  or  notes  using  the  on-line  notepad.  No  regard  was  given  to  the  type  of  notes  taken. 

Second,  three  note-type  groups  included  subjects  who,  regardless  of  the  notetaking  method  used  for  notetaking, 
took  verbatim,  paraphrased,  or  notes  using  their  own  notetaking  style.  The  three  note-type  groups,  of  course,  did  not 
contain  the  control  group. 

Notetaking  methods.  Those  subjects  who,  as  part  of  their  treatment,  took  notes  on-line,  used  a computerized 
notepad  which  could  be  opened  at  any  time  with  a single  keystroke.  Participants  could  simultaneously  view  a module 
screen  and  record  information  on  the  notepad.  The  computerized  notetaking  pad  consisted  of  a scrolling  window  that  had 
the  same  formatting,  editing,  printing,  and  copy/paste  capabilities  as  a word  processor.  The  notepad  was  also  dynamic;  it 
could  be  sized  and  repositioned  on  the  screen  at  any  time.  At  the  end  of  each  module  this  group  was  required  to  print  out 
a copy  of  their  notes  for  evaluation  by  the  researcher.  To  discourage  uncontrolled  review  of  notes,  subjects  were  asked 
not  to  save  their  notes  on  electronic  media  after  printing. 

Subjects  who  were  required  to  take  notes  using  pencil  and  paper  supplied  these  materials  for  recording  notes 
during  each  module.  They  also  submitted  their  notes  to  the  researcher  after  viewing  each  module. 

Monitoring  subject  participation.  Participants  were  instructed  to  complete  the  review  of  one  module  and  the 
related  notetaking,  if  assigned,  at  one  sitting.  Weekly  reports  were  obtained  from  the  AS/400  system  administrator  at  the 
university  which  listed  the  name  of  each  subject  who  accessed  the  required  module  during  the  assigned  period  and  the 
number  of  times  each  module  was  accessed  by  the  individual.  This  served  to  confirm  that  all  research  participants, 
including  the  control  group,  did  in  fact  view  the  assigned  modules  and  all  viewing  of  a module  was  done  in  a single 
session. 

Procedure 

Training  session.  Prior  to  the  actual  study,  each  research  subject  participated  in  a two-hour  training  session 
designed  to: 

• describe  the  study  and  its  purpose; 

• identify  the  responsibilities  of  each  participant  during  the  study; 

• review  the  criteria  and  benefits  of  good  notes  regardless  of  the  medium; 

• define  and  provide  examples  of  what  is  meant  by  paraphrased,  verbatim  notes,  and  individual  notetaking 
styles; 

• define  the  objectives  for  each  module  used  in  the  study; 

• provide  hands-on  experience  accessing  the  AS/400  tutorial  modules  and  the  on-line  notepad. 

Four  such  sessions  were  conducted  in  one  day.  Each  training  session  contained  subjects  who  had  been  assigned 
randomly  to  different  treatment  conditions.  Time  was  allowed  for  questions  and  clarification. 

Weekly  note  evaluation.  During  the  study,  all  groups  viewed  a different  module  each  week  for  four  weeks.  All 
subjects,  except  for  the  control  group,  took  notes  related  to  the  modules  using  their  assigned  method  and  note-type. 
Because  of  the  accessibility  of  computers  and  network  efficiency,  the  weekly  viewing/notetaking  could  be  completed  by 
the  subjects  at  any  time  during  the  week  assigned  to  that  module.  After  the  notes  were  completed,  they  were  submitted 
to  the  researcher  for  evaluation.  In  addition,  all  subjects  were  required  to  record  and  submit  their  start  and  stop  times  for 
viewing  and  taking  notes  from  each  module. 

The  weekly  evaluation  procedure  consisted  of  two  tasks.  First,  each  subject’s  notes  were  examined  to  determine 
if  they  were  produced  using  the  specified  note-type  (paraphrase,  verbatim,  or  own  style)  assigned  to  that  subject. 
Interestingly,  during  the  course  of  the  study,  all  subjects  recorded  notes  using  their  assigned  types.  Second,  each  subject’s 
notes  were  examined  to  determine  the  number  of  idea  units  and  to  identify  which  idea  units  were  present.  This 
information,  as  well  as  the  self-reported  time  used  to  view  and  take  notes  from  each  module,  was  recorded  weekly  for  each 
subject. 


Post- test.  One  week  following  the  completion  of  the  last  module,  an  exit  survey  and  post-test  were 
administered.  Prior  to  the  post-test,  the  subjects  had  an  open-ended  review  period.  During  this  period,  subjects  who  had 
taken  notes  had  their  notes  returned  and  were  allowed  to  review  them  for  a minimum  of  five  minutes  up  to  twenty-five 
minutes.  The  control  group  was  allowed  the  same  amount  of  time  for  mental  review.  The  time  used  by  each  subject  for 
review  was  recorded. 

Each  post- test  was  evaluated  and  the  following  scores  were  noted: 

• number  of  correctly  answered  factual-type  post-test  questions; 

• number  of  correctly  answered  synthesis-type  post-test  questions; 

• total  number  of  correctly  answered  questions; 

• recall  score  which  was  defined  as  the  correctly  answered  questions  which  had  corresponding  idea  units  recorded 
in  the  subject’s  notes. 


Results 

This  study  addressed  the  following  research  questions: 

• • does  using  an  on-line  computer  notetaking  device  during  computer-delivered  instruction  significantly  enhanced 
learning; 

• does  taking  paraphrased  notes  during  computer-delivered  instruction  promoted  greater  recall  than  verbatim 
notetaking  or  notes  taken  using  the  subject's  own  notetaking  style; 

• are  recall  scores  were  affected  significantly  by  the  method  and/or  type  of  notes  taken. 

This  study  was  a 2 x 3 factorial  design  with  a control  group.  The  independent  variables  are  the  notetaking  tool 
used  (notepad  or  pencil  and  paper)  and  note-type  (paraphrase,  verbatim,  or  own  notetaking  style).  The  control  group  took 
no  notes  while  viewing  the  computer-delivered  instruction. 

The  dependent  variables  include  total  post-test  score,  correctly  answered  post-test  factual  and  synthesis-type 
questions,  recall  score,  and  number  of  idea  units  recorded. 

Results  Related  to  Research  Question  1 

The  results  of  an  ANOVA  for  notetaking  method  groups  (C,  P'^,  OL)  and  total  post-test  score  variable  a 
difference  between  the  post-test  scores  of  the  group  members  (F(2,  109)  = 3.17,  p = .046)  (Table  1). 

Table  1 

ANOVA  for  Notetaking  Method  Groups  and  Total  Post-Test  Score  Variable 


Source 

df 

ss 

MS 

F 

p 

Between 

204.55 

102.27 

3.17 

.046* 

Groups 

2 

Within  Groups 

109 

3517.87 

32.27 

Total 

111 

3722.42 

*p  < .05 


Further  examination  using  the  Fisher  LSD  procedure  identified  a difference  between  the  control  group  and  the 
Gn-line  group's  post-test  scores. 

A one-way  ANOVA  revealed  no  difference  between  the  correctly  answered  post-test  synthesis-type  questions  of 
the  notetaking  method  groups  (C,  PP,  OL)  (F(2,  109)  = 0.36,  p = .698). 

A one-way  ANOVA  did  reveal  a difference  between  the  correctly  answered  post-test  factual -type  questions  (F(2, 
109)  = 3.74,  p = .027)  (Table  2). 


Table  2 

ANOVA  for  Notetaking  Method  Groups  and  Post-Test  Factual-Type  Question  Score  Variable 


Source 

df 

ss 

MS 

F 

p 

Between 

166.97 

83.48 

3.74 

.027* 

Groups 

2 

Within  Groups 

109 

2433.60 

22.33 

Total 

111 

2600.56 

*p  < .05 


A follow-up  Fisher  LSD  procedure  identified  a difference  between  the  way  the  control  group  and  the  on-line 
group  responded  to  factual-type  questions. 

Results  Related  to  Research  Question  2 

Two-way  ANOVAs  were  used  to  examine  whether  paraphrased  notes  promoted  greater  retention  than  verbatim 
notes  or  notes  taken  using  an  individuals  own  notetaking  style. 

The  total  post-test  scores,  post-test  synthesis-type  questions  scores,  and  post-test  factual-type  questions  scores  of 
the  notetaking  method  groups  ( PP,  OL)  and  the  note-type  groups  (O,  V,  P)  were  used  to  assess  the  ability  of  the  subjects 
to  retain  information. 

This  procedure  showed  that  there  were  no  differences  between  their  post-test  scores  (notetaking  method  groups  p 
= .150  and  the  note-type  groups  p = .306).  There  was  no  interaction  between  the  two  groups  (p  - .670). 

A 2 X 3 ANOVA  was  used  to  test  for  differences  between  the  post-test  synthesis  and  factual-type  question  scores 
of  the  notetaking  method  groups  (PP,  OL)  and  the  note-type  groups  ( O , V,  P) . This  procedure  showed  no  differences 
between  the  mean  post-test  synthesis-type  question  scores  of  the  notetaking  method  groups  (p  = .778)  or  the  note-type 
groups  (p  = .051).  There  was  no  interaction  between  the  two  groups  (p  = .735).  A two-way  ANOVA  showed  no 
significant  difference  between  the  post-test  factual-type  question  scores  of  the  notetaking  method  groups  (p  = .098)  or  the 
note-type  groups  (p  = .283).  There  was  no  interaction  between  the  two  groups  (p  = .651). 

Results  Related  to  Research  Question  3 

A 2 x 3 ANOVA  was  used  to  test  for  significant  differences  between  the  total  idea  units  of  the  notetaking 
method  groups  (PP,  OL)  and  the  note-type  groups  (O,  V,  P).  This  procedure  resulted  in  no  significant  differences 
between  the  mean  number  of  idea  units  recorded  by  the  notetaking  method  groups  (p  = .663)  and  the  note-type  groups  (p 
= .052).  There  was  no  significant  interaction  between  them  (p  = .879). 

A 2 x 3 ANOVA  was  used  to  test  for  differences  between  the  recall  scores  of  the  notetaking  method  groups  (PP, 
OL)  and  the  note-type  groups  (O,  V,  P) . This  procedure  indicated  that  there  was  a difference  between  the  recall  scores  of 
the  note-type  groups  (p  = .002)  but  no  difference  between  the  recall  scores  of  the  notetaking  method  groups  (p  - .064). 
There  was  no  interaction  between  the  groups  (p  - .616). 

Table  3 

Two-Way  ANOVA  between  Notetaking  Method  Groups  and  Note-Type  Groups  for  Recall  Score  Variable' 


Source 

df 

SS 

MS 

F 

P 

Explained 

5 

470.88 

94.18 

3.55 

.006 

Residual 

87 

2309.57 

26.55 

Total 

92 

2780.45 

30.22 

Source 

df 

SS 

MS 

F 

£ 

Not^taking  Method 

1 

93.68 

93.70 

3.53 

.064 

Note-Type 

2 

346.18 

173.09 

6.52 

.002* 

NM*NT 

2 

25.86 

12.93 

0.49 

.616 

* p < .05 


r r 
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A Fisher  LSD  procedure  was  run  on  the  total  recall  scores  of  the  note-type  groups.  The  results  indicate 
differences  between  verbatim  notetakers  and  both  notetakers  who  used  their  own  style  and  paraphrase  notctakers. 

A Pearson  Correlation  coefficient  between  recall  scores  and  post-test  scores  for  all  subjects  in  the  study  who 
took  notes  was  .6867  (pc.Ol). 

The  results  of  these  analyses  can  be  summarized  as  follows: 

1 . There  was  a difference  in  total  post-test  scores  between  the  control  group  which  took  no  notes  and  the 

treatment  group  which  took  notes  on-line.  The  difference,  which  supported  on-line  notetaking,  appeared  to 
be  due  to  the  way  the  control  and  on-line  notetakers  answered  the  factual-type  questions  on  the  post-test. 

2.  The  type  of  notes  taken  did  not  affect  the  overall  retention  of  information. 

3.  The  method  used  to  take  notes  and  the  type  of  notes  taken  did  not  affect  the  number  of  idea  units  recorded  by 

the  subjects. 

4.  A difference  in  recall  scores  was  found  between  the  verbatim  and  both  the  own  style  and  paraphrase 

notetakers.  It  supported  the  verbatim  notetakers. 

5.  There  was  a positive  correlation  between  the  recall  scores  and  total  post-test  scores  of  the  subjects. 

Discussion 


Research  Question  l 

It  was  predicted  that  computerized  notetaking  would  elicit  greater  learning  during  computer-delivered  instruction 
than  taking  notes  with  pencil  and  paper  or  taking  no  notes.  Comparisons  between  the  notetaking  method  groups’  total 
post-test  scores,  factual-and  synthesis- type  post-test  question  responses,  and  the  post-test  questions  associated  with  each 
module  were  used  to  assess  this  hypothesis. 

The  total  post-test  score  analysis  of  the  notetaking  method  groups  determined  a difference  did  exist  between  the 
post-test  scores  of  the  on-line  and  control  notetaking  methods  groups. 

Further  evaluation  also  revealed  an  overall  difference  in  the  number  of  correct  responses  to  factual-type  post-test 
questions  by  these  same  two  groups,  the  control  and  the  on-line,  while  no  difference  was  found  between  the  notetaking 
member  groups’  responses  to  the  synthesis- type  post-test  questions. 

Differences  in  total  post-test  scores.  Two  related  explanations  exist  as  to  why  subjects  who  took  notes  on-line 

performed  significantly  better  than  those  who  took  no  notes. 

The  first  explanation  is  related  to  the  length  of  the  study.  There  was  a period  of  five  weeks  between  the  time 
subjects  viewed  the  first  tutorial  module  and  the  administration  of  the  post-test.  Given  the  duration  of  the  study,  it 
should  not  be  surprising  that  taking  notes  and  reviewing  them  prior  to  testing  increased  retention  more  than  not  taking 
notes  and  reviewing  only  mentally.  This  finding  is  consistent  with  other  notetaking  research  which  concluded  that  taking 
and  reviewing  notes  was  more  beneficial  to  recall  than  taking  no  notes  and  using  only  a mental  review  to  prepare  for 
evaluation  (Kiewra,  1985,  1989;  Rickards  & Friedman,  1978). 

Second,  in  addition  to  providing  a method  to  record  notes  for  review,  taking  notes  using  the  on-line  notepad  may 
have  provided  subjects  with  a more  efficient  way  to: 

• include  information  in  notes; 

• organize  information  in  a meaningful  manner; 

• reduce  fatigue  which  may  occur  while  taking  notes  using  pencil  and  paper. 

These  potential  advantages,  in  turn,  may  have  encouraged  the  on-line  subjects  to  take  better,  more  succinct 
notes,  which  may  have  benefited  them  during  review  and  testing. 

Differences  in  factual  post-test  scores.  The  difference  in  the  total  post-test  scores  of  the  control  and  the  on-line 
groups  were  to  a great  extent  due  to  the  difference  in  the  way  group  members  answered  the  factual-type,  low-level 
questions  rather  than  the  high-level,  synthesis-type  questions.  Two  factors  may  have  influenced  the  subjects’  responses 
to  this  question  type.  First,  the  subjects  may  have  been  interacting  at  a novice  level  with  the  subject  matter  presented  in 
the  modules.  That  is,  they  may  have  lacked  the  subject  matter  depth  and  breadth  to  be  effective  at  integrating  prior 
knowledge  with  the  information  presented  in  the  tutorial  modules.  This  could  have  hampered  their  efforts  to  respond  to 
the  synthesis-type  questions.  The  subjects  may  have  been  more  efficient  and  effective  at  being  able  to  recall  information 
than  integrate  information,  and,  in  turn,  use  this  information  to  solve  problems. 

Second,  eighty  percent  of  the  post-test  questions  were  factual.  The  remaining  were  synthesis-type  questions. 
Perhaps  a more  equitable  number  of  both  question  types  would  have  presented  a clearer  picture  of  the  relationship,  if  any, 
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between  the  notetaking  methods  used  in  the  study  and  the  impact  of  these  methods  on  responses  to  factual  and  synthesis- 
type  post-test  questions  (Jonassen,  1984;  Rickards  & Friedman,  1978). 

Research  Question  2 

It  was  predicted  that  paraphrased  notes  taken  during  computer-delivered  instruction  and  reviewed  prior  to  testing 
would  promote  greater  recall  than  verbatim  notes,  or  notes  created  using  an  individual’s  own  notetaking  style. 

Paraphrased  notetaking  was  anticipated  to  be  a generative  activity  which  encouraged  subjects  to  consciously  and 
intentionally  relate  new  information  to  existing  knowledge,  thereby  enhancing  retention. 

To  evaluate  this  hypothesis,  the  note-type  groups’  total  post-test  scores,  factual-and  synthesis-type  post-test 
scores  were  examined. 

In  the  note-type  groups,  this  study  identified  no  differences  between  the  total  post-test  score,  factual-type  and 
synthesis-type  post-test  question  responses  of  the  subjects. 

These  findings  are  contrary  to  the  other  notetaking  depth  of  processing  research  (Shimmerlik  & Nolan,  1976; 
Bretzing  & Kulhavy,  1979;  Bretzing  & Kulhavy,  1981;  Jonassen,  1984).  Wittrock's  generative  learning  (1974,  1979) 
model  asserts  that  learners  who  actively  engage  in  building  associations  between  that  which  is  known  and  that  which  is 
to  be  learned  create  a greater  number  of  associations,  which,  in  turn,  enhances  recall. 

Lack  of  differences  in  total  post-test  and  factual-type,  and  synthesis-type  post-test  question  scores  by  the  note- 
type  groups.  Two  possible  explanations  exist  for  the  performance  of  the  note-type  groups  on  the  post-test  and  its 
component  question  types.  Although  the  notes  for  each  subject  were  examined  weekly  to  ensure  participants  recorded 
notes  that  were  either  paraphrased,  verbatim,  or  recorded  using  their  own  style,  no  determination  was  made  as  to  the 
amount  or  depth  of  processing  done  by  the  learner  producing  the  paraphrased  notes.  It  was  assumed  by  the  researcher  that 
additional  processing  took  place.  According  to  Wittrock’s  generative  hypothesis,  unless  the  learner  makes  use  of  his  or 
her  own  knowledge  for  interpreting  incoming  information,  the  process  is  not  generative.  Wittrock  contends  that  if  the 
processing  is  not  generative,  the  associations  that  are  formed  will  not  be  as  distinct,  so  less  meaning  will  be  generated 
from  the  information.  It  is  possible  that  when  paraphrasing  notes,  this  group  simply  did  not  engage  in  processing  to  the 
extent  suggested  by  Wittrock's  model. 

Second,  the  composition  of  the  post-test,  with  a large  percentage  of  factual-type  questions  and  limited  number  of 
synthesis-type  questions,  may  not  have  been  the  optimum  format  to  elicit  responses  which  assess  the  result  of  generative 
learning.  A free  recall  testing  format  or  additional  synthesis-type  questions  may  have  been  more  successful  at  assessing 
whether  paraphrased  notetaking  promoted  more  learning  than  other  forms  of  notetaking  (Jonassen,  1984;  Rickards  & 
Friedman,  1978). 

Research  Question  3 

It  was  predicted  that  information  present  in  notes  taken  during  computer-delivered  instruction  had  a significantly 
greater  chance  of  being  recalled  than  information  not  recorded  in  notes.  Previous  notetaking  research  has  concluded  that 
when  idea  units  which  serve  as  the  basis  for  post-test  questions  were  present  in  a subject’s  notes,  the  subject  had  a greater 
chance  of  recalling  that  information  than  information  not  recorded  in  notes  (Crawford,  1925b;  Howe,  1970;  Fisher  & 
Harris,  1973;  Kiewra  & Fletcher,  1984;  Bretzing  & Kulhavy,  1981).  Both  the  number  of  idea  units  and  recall  scores 
calculated  using  the  weekly  work  submitted  by  the  notetaking  method  groups  and  the  note-type  groups  were  used  to 
evaluate  this  hypothesis. 

No  differences  were  found  in  the  total  number  of  idea  units  recorded  by  the  members  of  the  notetaking  method 
groups  or  the  note-type  groups. 

Differences  in  the  recorded  idea  units  associated  with  the  following  modules  and  groups  were  found  between: 

• the  verbatim  notetakers  and  both  the  own  style  and  paraphrase  note-type  group  members  in  modules  one  and 
two; 

• the  on-line  and  pencil  and  paper  noletaking  methods  groups  members  in  module  four. 

The  purpose  of  examining  the  recall  scores  for  subjects  in  this  study  was  to  assess  what  impact  recorded  idea 
units  had  on  retention. 

Overall,  a difference  did  exist  between  the  total  recall  scores  of  the  verbatim  note-type  group  and  both  the  own 
style  and  paraphrase  note-type  groups  which  favored  the  verbatim  group. 

Lack  of  difference  in  total  idea  units.  Since  no  overall  difference  existed  in  the  number  of  idea  units  present  in 
the  notes  of  the  notetaking  method  groups  or  the  note-type  groups,  it  may  be  concluded  that  neither  the  methods  used  to 
take  notes  (on-line  notepad  or  pencil  and  paper)  nor  the  type  of  notes  taken  (own  style,  verbatim,  or  paraphrase) 
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influenced  the  number  of  idea  units  recorded  in  notes.  The  research  participants  were  apparently  able  to  identify  the 
important  ideas  as  well  as  record  them  regardless  of  the  method  or  type  of  notes  required. 

Although  there  were  no  differences  in  the  number  of  idea  units  among  the  groups,  there  were  differences  between 
the  recall  scores  of  the  verbatim  note-type  group  and  other  group  members.  Since  recall  scores  were  calculated  by 
determining  the  number  of  correctly  answered  post-test  questions  which  also  had  corresponding  idea  units  present  in  the 
subject’s  notes,  one  should  ask  what  the  verbatim  notetakers  included  in  their  notes  or  did  in  addition  to  recording  the  idea 
units  which  enhanced  their  recall.  The  answer,  perhaps,  is  in  the  very  nature  of  notes  taken,  namely  the  verbatim  notes. 

Subjects  who  were  asked  to  take  verbatim  notes  were  told  during  the  pre-study  training  session  that  the  notes 
should  contain  exact,  precise  information  about  the  content  of  the  module.  Such  structured  information  in  the  subjects 
notes  may  have  helped  them  review  more  effectively  for  predominantly  low-level,  factual  type  questions  because  their 
notes  had  the  correct,  detailed  information  for  review. 

Correlation  between  recall  score  and  post-test  score.  The  significant  positive  correlation  between  recall  scores 
and  post-test  scores  solidifies  the  importance  of  recording  notes  and  having  them  available  for  review. 

Recommendations  for  Further  Research 

Several  of  these  questions  suggest  opportunities  for  future  research.  They  include 

1.  What  factors  were  responsible  for  the  overall  success  of  the  on-line  notetaking  group?  Were  these  factors 
cognitively  based  or  were  they  associated  with  the  attributes  of  the  on-line  notepad  which  allowed  the  user  to  easily 
record,  organize,  and  read  information  once  it  has  been  recorded? 

2.  Is  there  one  type  of  notes  (i.e.  verbatim,  paraphrase,  or  own  style)  which  is  better  suited  for  low-level  or 
high-level  items?  Is  the  success  of  the  verbatim  notetakers  on  tests  of  factual  detail  due  to  the  process  of  recording 
information  or  due  to  the  completeness  of  the  notes  which  were  later  used  in  the  review  process? 

3.  What  level  of  computer  expertise  must  be  reached  before  an  on-line  notepad  can  be  used  easily  and 
confidently  by  the  learner?  Does  a subject's  attitude  towards  new  technologies  and  their  use  of  such  technologies  affect 
the  learner's  desire  to  use  an  on-line  notepad  for  notetaking? 

4.  At  what  age  can  a learner  effectively  take  notes  and  use  them  as  a useful  learning  strategy,  a resource  for 
study  and  review?  At  what  age  and  subject-level  mastery  are  learners  capable  of  generating  paraphrased  notes?  This  study 
assumed  that  the  subjects  knew  when  material  should  be  included  in  notes.  What  strategies  could  be  incorporated  into 
computer-delivered  instruction  to  alert  younger  learners  when  important  information,  which  perhaps  should  be  included  in 
their  notes,  is  presented  ? 

5.  Does  the  medium  used  to  deliver  instruction  affect  the  success  of  the  notetaking  method?  For  example, 
assume  that  the  content  of  the  tutorial  modules  u .ed  in  this  study  was  available  on  paper  as  well  as  on  the  computer. 

Will  on-line  notetaking  produce  similar  results  using  the  paper  format  as  it  did  using  the  computer? 

6.  By  allowing  subjects  to  both  take  and  review  their  own  notes,  this  research  supported  both  the  process  and 
product  functions  of  notetaking  during  computer -delivered  instruction.  Does  taking  notes  either  on-line  or  with  paper  and 
pencil  during  computer-delivered  instruction  benefit  students  who  are  allowed  to  only  record  notes  without  the 
opportunity  to  review  them? 

7.  Do  notetakers  outperform  non-notetakers  when  the  number  of  modules  and  the  time  required  to  complete 
them  is  either  increased  or  decreased?  Does  a delay  period  between  viewing  modules  significantly  affect  retention? 

8.  Given  that  an  on-line  notepad  is  required  for  on-line  notetaking,  are  there  elements  associated  with  the  on- 
line notepad’s  interface  design  which  could  be  modified  to  encourage  use  even  by  novice  computer  users? 

9.  There  has  been  no  research  done  which  assesses  the  merits  of  cooperative  notetaking  during  computer 
delivered  instruction.  Is  cooperative  notetaking  beneficial  to  those  involved?  Is  it  most  successful  when  notetaking  is 
viewed  as  a process,  a product,  or  a combination  of  both?  What  are  problems  that  arise  with  this  method?  Are  notes 
recorded  with  pencil  and  paper  or  on-line  prove  most  beneficial?  Do  cooperative  groups  encourage  learners  to  generate 
paraphrased  notes? 
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Abstract 


Computer  simulations  allow  students  to  explore  learning  environments  by  forming  hypotheses  and 
making  and  testing  predictions.  A dilemma  in  designing  such  simulations  is  how  to  provide  a challenging 
exploratory  environment  and  yet  provide  sufficient  student  support  so  that  students  do  not  become  lost. 

Directive  support  such  as  corrective  feedback  may  detract  from  the  exploratory  quality  of  a simulation.  Two 
methods  of  non-directive  support  are  1)  simplifying  the  initial  presentation  of  the  simulation  by  having  the 
student  manipulate  only  some  of  the  variables  and  2)  presenting  the  simulation  problem  to  a snu  ll  group  rather 
than  to  individuals  so  that  students  may  give  each  other  feedback  on  their  hypotheses  and  procedures . This  paper 
examL  les  the  effect  of  both  of  these  methods  and  their  interaction  upon  undergraduate  students  solving  a problem 
in  a four  variable  computer  simulation  of  the  spread  of  an  epidemic. 

Subjects  were  divided  into  four  treatment  groups:  individuals  working  with  reduced  initial  complexity , 
individuals  working  with  full  initial  complexity , groups  working  with  reduced  initial  complexity ; and  groups 
working  with  full  initial  complexity.  In  terms  of  the  effectiveness  of  instruction  as  measured  by  the  percentage 
of  students  reaching  a correct  solution,  a significantly  greater  proportion  of  subjects  who  worked  individually 
reached  the  correct  conclusion  when  the  task  was  initially  simplified.  In  contrast,  a greater  proportion  of 
subjects  who  worked  in  groups  reached  the  correct  conclusion  when  presented  with  full  initial  complexity. 

Subjects  performed  significantly  mon  simulation  runs  when  they  were  first  presented  fewer  variables 
and  finally  all  of  the  variables.  When  a task  is  complex  individuals  may  benefit  from  working  with  a group.  In 
this  study  group  work  provided  no  benefit  for  the  simplified  task.  The  study  suggests  it  may  be  preferable ; in 
presenting  a complex,  multi-variable  simulation , to  present  it  to  groups  for  solution  rather  than  attempt  to 
break  it  down  into  component  parts. 

With  increasing  emphasis  on  constructivist  approaches  to  learning  and  greater  availability  of  appropriate 
software,  instructional  computer-based  simulations  are  increasingly  being  used  to  allow  students  to  explore  complex 
multi-variable  phenomena  by  manipulating  variables  and  observing  changes.  For  example,  Smithtown  allows  students 
to  vary  economic  variables  (wages  and  prices)  and  observe  outcomes,  such  as  shift  in  the  demand  function  (Shute  & 
Glaser,  1990).  MIDAS,  a simulation  in  the  domain  of  decision-support  theory,  requires  users  to  see  the  effects  of 
specifying  different  weights  on  user  preference  scores  (de  Jong,  de  Hoog,  de  Vries,  1993).  REFRACT  allows  students  to 
see  the  effects  of  manipulating  variables  such  as  optical  density,  angle  of  refraction,  and  iir.^ge  distance  on  objects  such 
as  surfaces,  lenses  and  rays  (Reimann,  1991).  Numerous  such  simulations  now  exist  in  both  the  physical  and  social 
sciences.  Common  to  all  of  them  is  the  process  of  experimentation,  which  includes  forming  hypotheses,  making  and 
testing  predictions,  and  modifying  the  hypotheses.  A dilemma  is  that  this  process,  basic  to  scientific  inquiry,  is  both  a 
prerequisite  for  and  an  outcome  of  using  such  simulations.  Some  simulations  have  been  used  in  an  attempt  to  teach  the 
methods  of  scientific  inquiry,  for  example  Discovery  Lab  (Minnesota  Educational  Computing  Corporation,  1984).  But 
more  often,  as  in  the  examples  above,  the  main  purpose  is  to  teach  content,  namely,  the  relationships  of  variables 
defining  a physical  or  social  phenomenon. 

While  it  is  widely  believed  that  simulations  can  effectively  teach  about  complex  systems  through  this  controlled 
method  of  discovery,  studies  have  demonstrated  that  not  all  students  meet  with  success  (e.g.  Shute  & Glaser,  1990;  Njoo 
& De  Jong,  1991).  Learning  in  such  environments  requires  students  to  construct  their  own  knowledge  and  is  generally 
very  demanding  for  students,  even  with  some  form  of  instructional  support.  However,  the  provision  of  such  support 
needs  to  be  considered  in  terms  of  its  effects  on  the  nature  and  level  of  learners’  exploratory  strategies.  Ideally,  such 
support  should  facilitate  student  learning  while  maintaining  the  exploratory  nature  of  the  learning  experience.  More 
precisely,  what  is  needed  are  support  strategies  which  solve  the  dilemma  between  a low-support,  low  effectiveness , high 
involvement  and  high-support , high  effectiveness,  low  involvement  situation. 

Various  authors  have  investigated  the  effects  of  varying  levels  of  instructional  support  in  experiential  learning 
environments.  De  Jong,  de  Hoog,  de  Vries  (1993)  categorize  such  support  as  either  directive  (for  example,  hints, 
Socratic  dialogue,  corrective  feedback)  or  non-directive  (for  example,  hypothesis  sketch  pads,  goal  decomposition  trees, 
overviews  of  output  obtained  or  input  hisUry).  Non-directive  support  is  preferred  as  it  maintains  the  exploratory  nature 
of  the  learning  environment.  Shute  and  Glaser  (1990)  provided  non-directive  support  in  terms  of  tutoring  in  scientific 
skills.  Reimann  (1991)  provided  structure  for  subjects’  hypothesis  generation  and  testing  through  the  use  of  window  and 
notebook  facilities  to  keep  track  of  experiments  and  to  organize  and  manipulate  information,  and  through  the  use  of 
graphic  and  verbal  feedback.  Njoo  and  de  Jong  (1991)  studied  the  effects  of  providing  different  levels  of  support  to 
college  level  students  of  engineering  using  a simulation  in  the  domain  of  control  theory.  In  addition  to  providing 
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subjects  with  notetaking  facilities,  Quinn  and  Alessi  (1994)  showed  some  success  for  a strategy  of  breaking  the  overall 
task  into  subtasks  of  increasing  complexity  when  used  in  conjunction  with  a strategy  of  generating  and  testing  multiple 
hypotheses.  Quinn  & McCune  (1995)  showed  a greater  level  of  exploration  on  the  part  of  subjects  who  were  successful 
in  completing  a variable  optimization  task  than  those  who  were  unsuccessful. 

Our  current  focus  is  on  optimizing  student  learning  while  promoting  and  maintaining  exploratory  behavior  in 
the  context  of  a complex  multivariable  simulation.  The  literature  on  scientific  inquiry  provides  many  guidelines  to  foster 
hypothesis  generation  and  testing  and  subsequent  exploratory  behavior  (Popper,  1978;  Holland,  Holyoak,  Nisbett,  and 
Thagard,  1986),  with  recent  research  identifying  several  relevant  factors.  These  include  task  complexity  (Mynatt, 

Doherty,  & Tweney,  1978;  Gorman,  1989),  effects  of  working  in  groups  (Gorman,  1989),  structure  of  the  phenomenon 
(Klayman  & Ha,  1987),  goals  (Klayman  & Ha,  1987),  timing  of  tests  of  confirmation  and  falsification  (Mynatt  et  al, 
1978;  Tweney,  Doherty,  Womer,  Pliske,  Mynatt,  Gross,  & Arkelin,  1980),  familiarity  of  the  context  (Cheng  & 
Holyoak,  1985),  subjects*  prior  knowledge  (Klahr  & Dunbar,  1988),  and  the  number  of  hypotheses  generated  (Mynatt  et 
al,  1978;  Tweney  et  al,  1980;  Klahr  & Dunbar,  1988). 

This  present  study  is  concerned  with  the  development  of  non-directive  support  strategies  which  take  into  account 
two  of  the  factors  mentioned  above  - structure  of  the  phenomenon  and  the  effects  of  working  in  groups,  and  is  a follow- 
up to  Quinn  & Alessi  (1994).  In  that  study,  the  phenomenon  being  investigated  depended  on  several  variables  and  the 
overall  task  presented  to  subjects  was  to  investigate  the  effects  of  modifying  four  of  these  variables.  One  of  the  variables 
behaved  in  a counterintuitive  manner  (the  optimal  value  for  the  goal  assigned  to  subjects  was  its  maximum  value,  but  it 
appeared  to  many  subjects  that  the  optimal  value  should  be  the  minimum  value)  and  this  was  the  principal  reason  for 
subjects  failing  to  obtain  the  optimal  combination  of  the  four  variables.  The  purpose  of  this  study  was  to  increase  the 
proportion  of  subjects  who  obtained  the  optimal  combination  of  variables  by  requiring  subjects  to  work  in  dyads.  The 
rationale  behind  this  approach  was  that  requiring  subjects  to  work  in  pairs  would  increase  the  probability  of  subjects 
perceiving  the  counterintuitive  nature  of  the  optimal  combination  of  values,  while  maintaining  the  exploratory  nature  of 
the  task.  Thus,  requiring  subjects  to  work  in  pairs  was  seen  as  a form  of  non-directive  support.  In  addition,  some 
subjects  were  presented  with  the  overall  task  initially  and  some  subjects  were  presented  with  the  task  in  stages.  Such  a 
design  allowed  investigation  of  task  structure  on  performance.  Therefore,  the  principal  research  questions  in  this  study 
were:  1)  what  will  be  t!  e effect  of  subjects’  working  in  pairs  on  performance  on  a task  involving  variables  which  behave 
in  a counterintuitive  m mer?;  and  2)  what  is  the  effect  of  task  structure  on  such  performance? 

Methods 


Subjects 

Subjects  were  66  students  in  a teacher  education  program.  Twenty-one  students  were  enrolled  in  a 1 semester 
hour  course  titled  "Introduction  to  Microcomputing  for  Teachers”,  30  students  were  enrolled  in  a three  semester  hour 
course  titled  "Educational  Psychology  and  Measurement”,  and  the  remainder  (15)  were  enrolled  in  a three  semester  hour 
course  titled  "A  Survey  of  Computer  Applications  in  Instruction".  All  were  given  bonus  points  for  participation. 

The  Simulation 

The  computer  simulation  was  a model  of  the  spread  of  an  influenza  epidemic.  There  were  two  displays  in  the 
simulation  (Figure  1 and  2)  and  it  was  very  easy  to  operate.  On  the  control  display  (Figure  1)  the  subject  dragged  sliders 
to  choose  values  of  the  4 variables.  The  subject  then  pressed  the  Run  Simulation  button  to  obtain  the  next  display 
(Figure  2).  This  display  shows  both  tabular  and  graphic  representation  of  the  number  of  people  ill  across  time.  The 
subject  could  then  click  the  Simulation  Control  button  (returning  to  the  control  display)  to  try  new  values  and  observe 
the  output,  or  click  the  End  of  Phase  1 button  to  go  on  to  the  next  phase  of  the  experiment.  Typically,  subjects  go  back 
and  forth  between  these  two  displays,  changing  variables  and  observing  the  result,  until  they  believe  they  have  solved  the 
assigned  problem.  All  subjects  were  presented  with  the  following  goal:  to  determine  the  combination  of  variable  values 
that  would  keep  the  maximum  number  of  people  ill  in  any  one  week  as  low  as  possible.  Subjects  were  told  that  they 
could  consider  the  outbreak  to  be  over  as  soon  as  the  number  of  people  ill  fell  below  150  as  the  outbreak  receded. 
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Figure  1.  Simulation  control  display. 
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Figure  2.  Graphical  and  tabular  output  from  simulation. 


In  the  simulation  model,  the  number  of  people  ill  with  influenza  depended  on  four  variables:  the  number  of 
contacts  per  person  per  week,  the  time  to  illness,  the  duration  of  illness,  and  the  length  of  the  immune  period.  The 
output  observed  was  the  rise  and  decline  of  infection  within  a population  of  10,000  people  over  32  weeks.  The  range  of 
these  variables  are  presented  in  Table  1 . Table  2 presents  the  optimal  values  of  these  variables  for  each  of  the  goals 
presented  to  subjects.  In  Quinn  & Alessi  (1994),  the  proportion  of  subjects  who  reached  the  correct  conclusion  for  Goal 
1 was  approximately  70%.  In  addition,  the  principal -reason  for  an  incorrect  conclusion  to  Goal  1 was  an  incorrect 
specification  for  the  optimal  value  of  the  variable  “Time  to  Illness”  - 50.6%  of  subjects  who  failed  to  complete  the  task 
successfully  incorrectly  specified  the  minimum  value  of  the  variable  “Time  to  Illness”  rather  than  the  maximum  value. 


Table  1. 


Ranges  and  of  Variables  Affecting  Spread  of  Influenza  in  the 
Simulation  Model. 


Variable 

Range 

Number  of  Contacts  per 
Person  per  Week 

2 to  10  persons 

Time  to  Illness 

3 to  28  days 

Duration  of  Illness 

14  to  35  days 

Length  of  Immune  Period 

1 3 to  27  weeks 

Table  2. 

Optimum  Values  of  Variables  Affecting  Soread  of  Influenza 

in  the  Simulation  Model. 

Variable 

Optimal  values  for 
minimizing  maximum 
number  ill  in  any 
week. 

Optimal  values  for 
minimizing  the  number 
of  weeks  the  outbreak 
lasts. 

Number  of  Contacts  per 
Person  per  Week 

2 persons 

2 persons 

Time  to  Illness 

28  days 

2 days 

Duration  of  Illness 

14  days 

14  days 

Length  of  Immune  Period 

27  weeks 

27  weeks 

Design 

Subjects  were  divided  into  four  treatment  groups.  In  each  treatment  the  overall  goal  was  the  same  and  subjects 
were  required  to  manipulate  all  four  variables.  In  two  groups  (1234  and  14)  subjects  worked  on  their  own  and  in  two 
groups  (G234  and  G4)  students  worked  in  pairs.  All  subjects  were  randomly  assigned.  In  groups  14  and  G4,  subjects 
were  presented  with  all  four  variables  initially.  In  groups  1234  and  G234,  subjects  were  first  presented  with  two  variables 
and  were  required  to  specify  the  optimal  value  of  these  two  variables  (Phase  1).  When  subjects  concluded  they  had 
specified  the  correct  combination  of  variables,  a third  variable  was  introduced  and  the  process  was  repeated  for  three 
variables  (Phase  2).  Finally,  a fourth  variable  was  presented  and  subjects  determined  the  optimal  set  of  values  for  all  four 
variables  (Phase  3).  For  each  of  the  four  groups,  subjects  were  asked  to  complete  a questionnaire  about  their  interaction 
with  the  simulation.  The  dependent  variables  measured  were  the  proportion  of  subjects  who  reached  the  correct  solution 
(Correct)  and  the  amount  of  investigation  as  measured  by  number  of  simulation  runs  performed  by  subjects  (Simrun). 

Procedures 

Subjects  in  groups  14  and  1234  (who  worked  individually)  first  proceeded  through  an  orientation  phase  where 
they  were  introduced  to  the  subject  matter  of  the  simulation  and  learned  how  to  manipulate  the  simulation.  In  this  phase, 
subjects  were  also  given  a brief  description  of  the  four  variables.  Subjects  were  then  asked  to  generate  and  record  an 
initial  hypothesis  as  to  what  combination  of  variables  would  keep  the  number  of  weeks  the  outbreak  lasts  as  small  as 
possible,  while  at  the  same  time  keeping  the  maximum  number  ill  in  any  one  week  less  than  1500.  Subjects  could 
specify  a range  of  values  for  any  or  all  variables  and  were  also  given  the  option  of  indicating  that  they  did  not  think  that 
there  was  any  combination  of  variables  which  would  satisfy  the  goal.  Then,  subjects  were  instructed  to  begin 
investigation  by  manipulating  the  levels  of  the  variables  and  running  the  simulation.  Before  each  run  of  the  simulation, 
subjects  were  required  to  record  on  an  Experimentation  Record  Sheet  the  levels  of  each  of  the  variables  in  the  proposed 
simulation  run.  Then,  subjects  were  instructed  to  begin  investigation  by  manipulating  the  levels  of  the  variables  and 
running  the  simulation.  After  each  simulation  run,  subjects  were  asked  to  record  the  maximum  number  ill  in  any  one 


week  period  and  the  length  of  the  outbreak.  Subjects  were  not  given  feedback  as  to  the  correctness  of  the  levels  chosen. 
Only  by  manipulating  variables  and  observing  the  output  from  the  simulation  could  subjects  increase  their  confidence 
that  a particular  set  of  values  would  generate  data  to  fulfill  the  goals  of  the  simulation. 

Subjects  in  groups  G4  and  G234  proceeded  through  the  task  in  a manner  similar  to  subjects  in  groups  14  and 
1234  except  that  additional  instructions  were  given  about  working  together.  In  the  orientation  phase,  subjects  working  in 
pairs  were  told  that  they  would  be  working  on  the  task  together  and  were  instructed  to  proceed  through  the  orientation 
phase  working  together.  At  this  point,  subjects  were  also  told  that  throughout  the  study,  they  could  discuss  as  much  as 
they  wish  and  perform  as  many  simulation  runs  as  they  wished.  One  Experimentation  Record  Sheet  was  available  to 
each  group.  Subjects  were  told  to  decide  which  group  member  would  record  data  on  the  record  sheet  and  which  member 
would  interact  with  the  simulation.  Finally,  it  was  emphasized  to  subjects  that  while  they  would  work  in  groups,  it  was 
not  necessary  that  they  agree  on  the  optimal  combination  of  values  for  the  specified  goal.  Each  group  member  recorded 
their  conclusions  separately. 


Results 

Analysis  of  variance  was  first  performed  to  determine  the  effectiveness  and  efficiency  of  the  two  treatments  - 
Learning  Context  (individual  versus  group  work),  and  Task  Structure  (presentation  of  variables  all  at  once  or  in  parts). 
This  was  done  by  analyzing  the  dependent  variables  Correct  - proportion  of  subjects  reaching  the  correct  conclusion,  and 
Simrun  - the  number  of  simulation  runs  performed  by  subjects. 

Effectiveness 

The  dependent  variable.  Correct,  measured  the  proportion  of  subjects  who  came  to  the  correct  conclusion. 
Subjects  who  came  to  the  correct  conclusion  were  given  a score  of  1 ; subjects  who  did  not  reach  the  correct  conclusion 
were  given  a score  of  0.  Table  3 presents  the  mean  proportion  and  standard  deviation  of  students  who  reached  the  correct 
conclusion  in  the  simulation  in  each  group.  Analysis  of  variance  indicated  a significant  Learning  Context  by  Task 
Structure  interaction  [F(l,6l)  = 5.330,  MSerror  = 1.224,  p = .024].  The  data  are  plotted  for  clarity  in  Figure  3.  In  view 
of  the  interaction,  simple  effects  were  examined  at  each  level  of  Learning  Context.  The  test  for  simple  effects  of 
Learning  Context  indicated  a significant  effect  for  Learning  Context  only  in  the  case  where  all  four  variables  influencing 
the  outcome  of  the  simulation  were  presented  at  once  [F(l,61)  = 20.436,  MSerror  = 4.694,  p = .000].  When  all  four 
variables  were  presented  at  once,  subjects  performed  better  when  they  worked  in  a group  than  when  they  worked  alone. 
The  test  for  simple  effects  of  Task  Structure  indicated  a significant  effect  for  Task  Structure  when  subjects  worked 
individually  [F(l ,61 ) = 38.773,  MSerror  = .230,  p = .000]  and  in  groups  [F(l,61)  = 8.845,  MSerror  = .230,  p = .004.  A 
significantly  greater  proportion  of  subjects  who  worked  individually  reached  the  correct  conclusion  when  the  task  was 
presented  in  parts;  in  contrast  a greater  proportion  of  subjects  who  worked  in  groups  reached  the  correct  conclusion  when 
the  task  was  presented  in  its  entirety. 
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Figure  3. 


Proportion  of  subjects  reaching  the  correct 
conclusion  by  learning  context  and  task  structure. 


Proportion 

Correct 


Table  3. 

Proportion  of  Subjects  Reaching  the  Correct  Conclusion. 


GROUP 

Proportion  of  Subjects  Reaching  Correct 
Conclusion 

Group  1234 

M 

.444 

SD 

.511 

(n=18x 

Group  G234 

M 

.278 

SD 

.461 

(n=18) 

Group  14 

M 

.400 

SD 

.507 

(n=15) 

Group  G4 

M 

.786 

SD 

.426 

(n=14) 
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Efficiency 

The  dependent  variable  Simrun  measured  the  number  of  simulation  runs  performed  by  each  subject.  Table  4 
presents  the  means  and  standard  deviations  for  Simrun  only  for  those  subjects  who  successfully  completed  the 
simulation.  Analysis  of  variance  did  not  indicate  any  significant  differences  between  groups  in  terms  of  efficiency. 

Table  5 presents  the  means  and  standard  deviations  for  Simrun  for  all  groups.  Analysis  of  variance  indicated 
significant  main  effects  for  Task  Structure  [F(l,61)  = 12.644,  MSerror  = 221.880,  p = .001],  Subjects  performed 
significantly  fewer  simulation  runs  when  the  task  was  presented  all  at  once  rather  than  in  parts. 

Table  4. 

Number  of  Simulation  Runs  Performed  bv  Subjects  Reaching  the  Correct  Conclusion, 


GROUP 

Number  of  Simulation  Runs  Performed  by 
Subjects  Reaching  the  Correct  Conclusion 

Group  1234 

M 

31.835 

SD 

12.722 

(n=8) 

Group  G234 

M 

36.800 

SD 

21.241 

(n=5) 

Group  14 

M 

30.107 

SD 

20.203 

(n=6) 

Group  G4 

M 

18.091 

SD 

11.077 
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Table  5. 

Number  of  Simulation  Runs  Performed  bv  All  Subjects. 


GROUP 

Number  of  Simulation  Runs 
Performed  by  All  Subjects 

Group  1234 

M 

30.111 

SD 

13.248 

1D=18) 

Group  G234 

M 

35.444 

SD 

16.343 

- 

(n=18) 

Group  14 

M 

20.400 

SD 

16.008 

(n=15) 

Group  G4 

M 

18.714 

SD 

13.652 

(n=14) 
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Discussion 


With  regard  to  learning  effectiveness,  the  interaction  effects  indicated  no  clear  advantage  for  either  Learning 
Context  or  Group  Structure  alone.  Thus,  there  was  no  clear  advantage  for  presenting  the  task  in  parts  or  allowing 
subjects  to  work  with  a partner.  Rather,  the  results  indicated  that  the  effectiveness  of  allowing  subjects  to  work  with  a 
partner  depended  on  whether  all  variables  were  presented  at  once  or  in  parts.  The  findings  suggest  that  subjects  who 
worked  with  a partner  performed  better  when  the  task  was  presented  all  at  once  than  when  it  was  presented  in  stages. 

This  finding  suggests  that  when  a task  is  complex,  individuals  may  benefit  from  working  with  a partner,  but  when  a task 
is  simple  there  may  be  no  advantage  to  working  with  a partner.  On  the  other  hand,  subjects  who  worked  alone  performed 
better  when  the  task  was  presented  in  parts  rather  than  all  at  once.  Thus,  our  hypothesis  that  subjects  in  the  group 
condition  would  outperform  those  working  individually  was  not  confirmed.  In  addition,  there  was  no  clear  advantage  for 
breaking  the  task  into  subtasks. 

It  was  not  surprising  that  groups  performed  better  when  all  variables  were  presented  at  once.  We  might  have 
concluded  that  the  simple  task  didn't  require  the  extra  resources  of  a group  if  both  the  individuals  and  groups  had  done 
very  well,  but  in  fact  both  did  rather  poorly,  especially  the  groups.  This  may  be  due  to  the  fact  that  presenting  the 
simulation  in  stages  of  increasing  complexity  made  it  logistically  more  complicated.  Students  may  have  expended  a 
great  deal  of  effort  on  the  first  sections  and  then  may  not  have  wanted  to  devote  much  of  their  time  or  attention  to  the 
final  stage  to  reach  a correct  conclusion.  Groups  did  not  make  significantly  more  ions  but  they  may  have  taken  a bit 
longer  per  run  especially  in  the  simplified  presentation.  Thus  the  groups  who  were  presented  with  the  task  in  parts  may 
have  become  fatigued  toward  the  end  of  the  simulation. 

Another  major  finding  in  this  study  was  that  subjects  who  worked  individually  performed  significantly  better 
when  the  task  was  presented  in  parts.  One  implication  of  this  finding  is  that  when  subjects  work  alone  they  do  not  have 
access  to  the  types  of  support  provided  in  a group  setting,  thus  they  benefit  from  instructional  support  built  into  the 
instruction.  Overall  the  findings  indicate  that  there  are  two  ways  to  provide  instructional  support.  First,  the  task  may  be 
divided  into  more  manageable  parts  or  individuals  may  be  allowed  to  work  with  a partner;  however  both  of  these 
strategies  or  types  of  nondirective  support  should  not  be  combined.  Also,  although  learning  is  improved  by 
decomposing  a task  for  individuals  working  alone,  the  far  more  effective  strategy  is  to  present  the  task  all  at  once  but  to 
groups  rather  than  to  individuals. 

In  terms  of  learning  efficiency,  the  results  of  this  study  indicated  an  advantage  for  presenting  the  task  in  its 
complexity.  Subjects  performed  significantly  fewer  runs  when  they  were  presented  with  all  four  variables  at  once.  This 
finding  coupled  with  the  finding  regarding  learning  effectiveness  suggests  that  learning  is  most  effective  and  efficient 
when  tasks  are  presented  in  all  their  complexity  to  groups  as  opposed  to  individuals.  One  implication  of  this  result  is 
that  attempting  to  simplify  a task  by  decomposing  it  into  parts  may  make  the  task  more  logistically  cumbersome. 
Another  interpretation  of  this  finding  is  that  breaking  a complex  task  into  its  component  parts  and  making  it  more 
cognitively  manageable  for  learners  may  increase  their  willingness  to  persevere  on  a task. 

Conclusion 

This  study  investigated  the  effects  of  group  versus  individual  learning  and  task  presentation  format  on  learners' 
ability  to  understand  and  simultaneously  consider  the  effects  of  more  than  one  variable  on  a single  dependent  variable. 
There  appears  to  be  no  clear  advantage  for  individual  versus  group  work.  The  effectiveness  of  either  learning  context 
depends  on  the  complexity  of  the  task  at  hand.  When  the  task  is  simple,  learning  context  does  not  make  a difference  in 
performance.  However,  when  the  task  is  complex  there  is  a significant  advantage  to  working  in  a group.  Some  of  the 
benefits  of  working  with  other  individuals  on  a task  are  that  it  encourages  one  to  view  phenomena  from  multiple 
perspectives,  increases  level  of  commitment  to  solving  a task,  and  also  increases  the  amount  of  cognitive  effort  invested 
in  a task  because  of  the  subtle  social  pressure  of  working  in  a group  (Rysavy  & Sales,  1991).  Thus,  individuals  are 
more  likely  to  reap  the  benefits  of  working  in  groups  on  complex  tasks  than  on  simple  tasks,  since  simple  tasks  do  not 
require  viewing  ideas  from  multiple  perspectives,  or  investing  a great  amount  of  cognitive  effort,  or  an  inordinate  amount 
of  perseverance.  The  findings  of  this  study  suggest  that  a more  effective  way  of  facilitating  learning  of  complex  tasks  is 
to  have  individuals  work  in  groups  rather  than  simplify  the  task  by  decomposing  it  into  more  manageable  pieces.  This 
finding  is  encouraging  in  light  of  the  practical  concerns  of  most  teachers.  Rather  than  spend  time  watering  down  difficult 
concepts  teachers  would  do  better  to  simply  have  students  work  with  a partner  to  solve  a difficult  task  or  understand  a 
complex  topic.  This  strategy  does  not  require  more  time  on  the  part  of  the  teacher  and  also  has  numerous  other  benefits 
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for  the  student  apart  from  the  cognitive  advantages,  such  as  the  acquisition  and  practice  of  social  skills,  improved  self 
esteem,  and  the  increase  in  perceived  status  of  low  achieving  students  (Rysavy  & Sales,  1991). 

Given  the  increasing  emphasis  on  constructivist  software  tools  such  as  simulations,  it  is  imperative  that 
educators  identify  ways  to  help  students  manage  the  cognitive  demands  of  computer-based  simulations  yet  still  preserve 
their  exploratory  nature.  This  study  suggests  that  one  way  to  facilitate  learning  from  simulations  which  present  complex 
tasks  is  to  allow  subjects  to  work  in  groups.  However,  this  study  did  not  assess  comprehension  of  the  material  presented 
in  the  simulation,  thus,  one  issue  that  needs  to  be  addressed  in  future  research  is  comprehension  as  opposed  to  simply 
arriving  at  the  correct  conclusion  in  a simulation  or  not.  Thus,  future  research  should  assess  comprehension  through  the 
use  of  a posttest  given  that  it  is  entirely  reasonable  that  subjects  may  have  learned  a great  deal  yet  may  still  not  have 
arrived  at  the  correct  conclusion. 

A second  issue  that  needs  to  be  addressed  in  future  research  concerns  the  optimal  size  of  groups.  At  what  point 
do  the  logistics  of  additional  group  members  outweigh  the  pedagogical  benefits  of  collaborative  learning  in  a simulation? 
Future  research  should  investigate  the  effects  of  allowing  subjects  to  work  in  groups  consisting  of  more  than  two 
members. 

A third  issue  for  future  research  concerns  the  operational  definition  of  task  complexity.  In  this  study,  task 
complexity  was  defined  in  terms  of  logistics.  That  is,  the  task  was  simplified  by  decomposing  it  into  parts  as  opposed 
to  presenting  a different  task  that  was  conceptually  simpler.  It  would  be  of  interest  to  investigate  the  interaction  between 
task  complexity  when  the  task  is  made  simpler  conceptually  as  well  as  logistically. 
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Background  and  Rationale 

During  years  of  teaching  principles  of  instructional  design  (ID)  to  graduate  and  undergraduate  education  majors, 
the  authors  have  noted  that,  when  initially  presented  with  models  that  use  rectangles  and  straight  lines  to  visualize  the 
process  (e.g.  Dick  & Carey,  1990;  Seels  & Glasgow,  1990;  Smith  & Ragan,  1993),  students’  first  impression  of  the  ID 
process  is  that  it  is  rigid,  inflexible,  fixed,  and  perhaps  not  very  relevant  for  use  in  a real-world  K-12  classroom. 

However,  when  students  have  first  been  introduced  to  Kemp’s  circular  model  (Kemp,  1985;  Kemp  et  al.,  1994),  students’ 
initial  perceptions  of  the  ID  process  are  that  the  process  is  somewhat  flexible  and  adaptive  and  may  be  beneficial  to  them 
as  teachers.  Kemp  himself  indicates  (Kemp,  1985,  p.  12;  Kemp  et  al.,  1994,  p.  10)  that  his  choice  of  a circular  model 
was  chosen  to  visually  emphasize  the  flexibility  of  his  approach  to  instructional  design. 

From  the  perspective  of  visual  communication,  vertical  and  horizontal  lines,  squares,  and  blocks  are  associated 
with  feelings  of  stability  and  stasis  while  circular  forms  and  curved  lines  imply  movement,  motion,  and  dynamism 
(Dondis,  1973;  Heinich,  Molenda,  & Russell,  1993).  If,  when  working  to  encourage  teachers  to  use  an  instructional 
design  model  as  a flexible  framework  for  designing  good  instruction  and  not  as  a lock-step,  rigid  format  to  be  followed 
without  integrating  one’s  own  professional  expertise,  then  the  manner  in  which  an  instructional  design  model  is  visually 
depicted  becomes  an  important  instructional  consideration.  Students’  perceptions  of  the  instructional  design  process  itself 
may  be  heavily  influenced  by  the  way  in  which  ED  models  are  visually  depicted. 

In  thinking  about  students*  potential  perceptions  of  both  the  process  of  instructional  design  and  the  models  used 
to  visualize  it,  the  authors  identified  three  factors  of  primary  interest:  flexibility,  organization,  and  value.  Flexibility, 
used  in  the  context  of  this  study,  is  the  characteristic  of  the  ID  process  or  model  that  indicates  the  degree  to  which  it  is 
responsive  to  being  adapted  or  changed.  Organization,  as  defined  by  the  authors,  is  the  characteristic  of  the  ID  process  or 
model  that  indicates  the  manner  of  relationship  among  the  process/model  elements,  the  overall  structure  and  pattern  of  the 
process/model,  and  the  logic,  meaning,  and  clarity  of  that  pattern.  Value  is  the  characteristic  of  the  ID  process  or  model 
that  indicates  the  degree  of  usefulness  or  importance. 

This  is  the  seminal  investigation  in  a planned  series  of  investigations  and,  as  such,  the  major  goal  was  to 
determine  if  the  way  in  which  the  instructional  design  process  is  visually  depicted  by  a two-dimensional  model  influences 
preservice  teacher  perceptions  of  the  flexibility,  organization,  and  value  of  the  ED  process  itself. 

Research  Questions 

As  indicated  above,  this  study  was  designed  to  investigate  the  influence  of  the  visual  display  of  an  instructional 
design  mode!  on  preservice  teachers’  perceptions  of  the  instructional  design  process.  The  overall  research  hypotheses 
included  the  following: 

1.  Providing  preservice  teachers  with  information  about  the  instructional  design  process  will  increase  their 
perceptions  of  the  flexibility,  organization,  and  value  of  the  process. 

2.  Preservice  teachers  will  perceive  the  ID  process  to  be  more  flexible  when  a curved/oval  model  is  used  to 

visually  represent  the  process  than  when  a straight/rectangle  model  is  used. 

3.  Preservice  teachers  will  perceive  the  ID  process  to  be  no  more  or  no  less  organized  when  a curved/oval  model 

is  used  to  visually  represent  the  process  than  when  a straight/rectangle  model  is  used. 

4.  Preservice  teachers  will  perceive  the  ID  process  to  be  more  valuable  when  a curved/oval  model  is  used  to 

visually  represent  the  process  than  when  a straight/rectangle  model  is  used. 

5.  Prior  experience  with  ID,  prior  experience  teaching,  anticipated  teaching  level  or  content,  year  in  college,  and 

gender  will  have  no  effect  upon  preservice  teachers’  perceptions  of  the  flexibility,  organization,  or  value  of 
the  ID  process  as  visually  depicted  by  either  curved/oval  or  straight/rectangle  models. 

Additionally,  the  authors  were  interested  in  assessing  preservice  teachers’  general  reactions  to  the  instructional  design 
process.  Study  participants  were  asked  to  respond  to  the  following: 

6.  List  any  other  words  to  describe  how  you  feel  about  the  instructional  design  process. 

7.  Which  model  do  you  think  best  represents  what  happens  in  the  instructional  design  process? 

8.  If  you  were  going  to  teach  some  of  the  principles  of  the  instructional  design  process  to  someone  else  who 

was  planning  to  be  a teacher,  which  model  would  you  show  to  the  other  person  as  you  explained  the 
instructional  design  process? 


Methodology 

This  study  was  conducted  during  the  Fall  1995  semester  following  a pilot  study  that  occurred  during  the  Spring 
1995  term.  In  both  cases,  the  sample  consisted  of  undergraduate  education  majors  enrolled  in  an  introductory  education 
class  at  a four-year  university  in  the  Rocky  Mountain  west. 
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Word  Pairs: 

Flexibility  Characteristic 


structured unstructured 

inflexible flexible 

rigid adaptable 

firm malleable 

regulated sporadic 

unyielding yielding 

exacting modifiable 


fixed 

constant  — 


pliable 

—changing 


Figure  1.  Flexibility  antonym  pairs 


The  students  were  first  assessed  on  their  initial  knowledge  and  perceptions  of  the  ID  process.  The  assessment 
included  a self-report  of  demographic  and  experiential  characteristics  including  prior  experience  with  instructional  design 
(“none”  or  “some”)  and  a 26-item,  5-point  Likert-type  scale  on  which  students  indicated  the  degree  to  which  they  felt  one 
or  the  other  of  a particular  set  of  antonyms  described  their  feelings  about  the  ID  process  (See  Figures  1,2,  and  3).  These 
antonym  sets  were  categorized  into  three  characteristic  areas:  flexibility,  organization,  and  value/usefulness.  Students 
were  then  given  a three  hour  introduction  to  instructional  design  as  part  of  a standard  curriculum.  During  this  period  of 
direct  oral  instruction  over  the  material,  students  received  a printed  outline  of  the  “steps”  in  the  Smith-Ragan  (1993)  ID 
model  but  no  visual  representation  of  the  model.  After  instruction,  students’  perceptions  of  the  ID  process  again  were 
assessed  using  the  26-item  Likert-type  scale.  After  this  posttest,  student  viewed  first  one  and  then  the  second  of  two 
visual  depictions  of  the  Smith-Ragan  ID  model.  One  model  was  drawn  with  curved  lines  and  ovals  and  the  other  original 
model  was  formed  with  straight  lines  and  rectangles.  The  layout,  size,  font  styles,  and  other  visual  elements  remained 
constant  between  the  two  models  (See  Figures  4 and  5). 


Word  Pairs: 


Organization  Characteristic 


linear circular 

abstract concrete 

random — sequential 

inconsistent consistent 

informal formal 

irrational rational 

misleading clear 

complicated simple 

disorganized — organized 

unplanned planned 

haphazard methodical 

disorderly orderly 


Figure  2.  Organization  antonym  pairs 
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Word  Pairs: 
Value  Characteristic 

unhelpful helpful 

difficult easy 

confusing logical 

foolish sensible 

worthless valuable 


Figure  3.  Value  antonym  pairs 


During  the  study,  half  of  the  students  were  given  the  curved/oval  model  first  and  then  completed  the  26-item 
assessment  and  one  question  probing  for  any  other  words  they  associated  with  instructional  design  process.  The  students 
then  viewed  the  straight/rectangle  model  and  answered  the  same  26  response  items  and  follow-up  question  while  referring 
to  the  second  visual.  The  remaining  half  of  the  students  were  presented  with  the  visual  models  in  the  reverse  order.  After 
viewing  both  models,  all  students  were  ask  to  specify  which  model  (curved  or  straight)  best  represented  what  “happens” 
in  the  instructional  design  process  and  which  of  the  two  models  they  would  use  if  they  were  teaching  the  ID  process  to 
someone  else  who  was  planning  to  be  a teacher. 

Thirty-six  (36)  students  participated,  of  which  24  (67%)  were  female  and  12  (33%)  were  male.  As  a group, 
participants  had  completed  an  average  of  1 3.92  years  of  schooling  and  represented  a wide  range  of  teaching  levels  and 
content  areas.  Fourteen  students  (39%)  had  taught  in  some  capacity  before,  whether  in  a traditional  classroom  or  as  an 
aide,  coach,  community  service  volunteer,  or  religious  education  instructor.  None  had  any  experience  with  the 
instructional  design  process. 
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INSTRUCTIONAL  DESIGN  PROCESS* 


Model  A 


ANALYSIS 


•Adapted  from  Smith,  P.  L.  & Ragan,  T.  J.  (1993)  Instuctional  Design.  New  York:  Merrill 

Figure  4.  Curved/oval  ID  model 


GOo 


586 


INSTRUCTIONAL  DESIGN  PROCESS* 


f 


Model  B 


ANALYSIS 


^Adapted  from  Smith,  P.  L.  & Ragan,  T.  J.  (1993)  Instuctional  Design.  New  York:  Merrill 

Figure  S.  Straight/rectangle  ID  model 


Results 

The  first  four  research  questions  were  examined  using  a paired  two-sample  for  means  t-test  to  indicate  any 
change  in  response  before  and  after  instruction  and  also  before  and  after  viewing  of  the  ID  models.  This  test  was 
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significant  at  the  .05  level  for  Question  One.  Data  suggest  that  preservice  teachers  did  show  a change  in  their  responses 
between  the  pretest  and  the  posttest  assessing  their  perceptions  of  the  flexibility,  organization  and  value  of  the 
instructional  design  process,  though  current  data  do  not  yet  suggest  where  these  changes  occurred.  On  the  posttest, 
respondents  were  highly  consistent  in  choosing  the  terms  sensible  and  valuable  to  describe  the  ID  process.  Both  terms 
had  means  in  excess  of  4.5  on  a 1 to  5 scale  with  the  higher  numeric  values  being  associated  with  those  two  terms. 
Questions  2 through  4,  assessing  the  flexibility,  organization,  and  value  of  each  of  the  two  models,  had  varied  results. 
Respondents  found  the  curved/oval  model  to  be  significantly  more  flexible  at  the  .05  level  when  it  was  the  second  model 
they  viewed.  No  significant  preference  existed  when  the  curved/oval  model  was  viewed  first.  Participants  perceived  the 
straight/rectangle  model  to  be  more  organized  (significant  at  the  .05  level)  regardless  of  viewing  order.  No  significant 
difference  in  the  models  was  found  regarding  value/usefulness. 

In  regard  to  Question  5,  the  demographic  and  experiential  characteristics  selected  did  not  satistically  influence 
responses  to  any  of  the  questions  posed.  Analysis  of  open-ended  reponses  also  revealed  no  patterns  or  trends  related  to  the 
selected  demographic  or  experiential  charateristics. 

In  analyzing  the  open-ended  responses  to  Questions  6 through  8,  interesting  patterns  emerged.  Student 
responses  to  Question  6 are  summarized  in  Figure  6. 

Model  Viewed 


o □ 


• Good  framework 

• Easier  to  comprehend 

• Worthwhile  process 

• Smooth 

• Orderly 

• Confusing 

• Very  prepared 

• Easier  to  follow 

• Cyclical  effect 

• Constant  learning  process 

• Learning 

• Adaptable,  but  structured 

• Out  of  order 

• Time  consuming,  but  valuable 

• Overwhelming 

• Strict 

• Too  many  paths 

•Straightforward 

• Confusing 

• Arranged 

• Lost  running  in  circles 

• Structured 

• Cognitive  flow  chart 

• Square 

• Too  concrete  for  my  thinking  style 

• Orderly 

« • Overwhelmed 

• Direct  approach 

• Difficult  to  follow 

• Learning 

• Chaotic 

• Revise 

• Confusing 

• Looks  complex,  but  isn’t 

• Flows  smoothly 

• Organized 

• Organized,  but  adaptable 

• Visually  clear 

• Not  enough  flexibility 
•Time  consuming,  but  valuable 

• Understandable 

Figure  6.  Words  generated  to  describe  the  ID  process 


Though  data  were  summarized  in  a manner  to  observe  any  influence  caused  by  the  order  in  which  the  students 
viewed  the  models,  presentation  order  does  not  appear  to  influence  these  responses.  Student-generated  words  used  to 
describe  the  instructional  design  process  as  depicted  in  the  curved/oval  model  included  the  terms  orderly,  confusing, 
smooth,  overwhelming,  adaptable  but  structured,  time  consuming  but  valuable,  out  of  order,  easier  to  comprehend,  and 
easier  to  follow.  Quite  similarly,  student  generated  words  used  to  describe  the  instructional  design  process  as  depicted  in 
the  straight/rectangle  model  included  the  terms  orderly,  confusing,  flows  smoothly,  overwhelmed,  organized  but 
adaptable,  time  consuming  but  valuable,  chaotic,  understandable,  and  straightforward.  Overall,  the  very  same  words  or 
those  with  similar  meanings  were  used  to  describe  both  the  positive  and  negative  aspects  of  the  instructional  design 
process  regardless  of  whether  curved  or  straight  lines  were  used  to  visually  depict  the  model.  These  descriptive  words 
were  not  necessarily  provided  by  the  same  students. 

In  considering  Question  7,  Figure  7 depicts  the  number  of  students  who  indicated  a preference  for  the 
curved/oval,  the  straight/rectangle,  or  neither  model  as  the  best  representation  of  what  occurs  during  the  instructional 
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design  process.  Descriptive  statistics  indicate  that,  of  the  36  preservice  teachers,  13  students  (36%)  believed  that  the 
curved/oval  model  best  represented  the  ID  process,  14  students  (39%)  believed  that  a straight/  rectangular  models  best  did 
so,  and  9 students  (25%)  believed  neither  model  was  better. 
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Model  Viewed  First 
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Figure  7.  Number  of  students  selecting  each  model  as  “best” 


When  the  curved/oval  model  was  viewed  first  by  18  students,  9 students  (50%)  indicated  that  it  was  the  best 
representation  of  the  ID  process,  4 students  (22%)  indicated  that  the  straight/rectangle  model  was  the  best,  and  5 students 
(28%)  indicated  neither  was  better.  When  the  straight/rectangle  model  was  viewed  first  by  18  students,  1 1 students  (61%) 
indicated  that  the  curved/oval  model  was  the  best,  3 students  (17%)  indicated  that  the  straight/rectangle  model  was  best, 
and  4 students  (22%)  indicated  that  neither  was  better.  Overall,  20  students  (56%)  indicated  that  the  curved/oval  model 
was  the  best  representation  of  what  happens  in  the  instructional  design  process,  7 students  (19%)  indicated  that  the 
straight/rectangular  model  was  the  best,  and  9 (25%)  students  responded  that  neither  model  was  a better  representation  of 
the  ID  process. 

Data  relating  to  Question  8 above  are  presented  in  Figure  8.  In  a similar  manner  to  the  data  collected  for 
Question  7,  descriptive  statistics  indicate  that  of  the  36  preservice  teachers,  14  students  (39%)  believed  that  they  would 
use  the  curved/oval  model  to  teach  the  ID  process,  13  students  (36%)  believed  that  they  would  use  the  straight/rectangle 
model  to  do  so,  and  9 students  (25%)  had  no  preference  for  use.  When  the  curved/oval  model  was  viewed  first  by  18 
students,  9 students  (50%)  indicated  that  it  was  the  model  they  would  use  to  teach  the  ID  process,  5 students  (28%) 
indicated  that  they  would  use  the  straight/  rectangle  model,  and  4 students  (22%)  responded  that  they  had  no  preference  for 
use.  When  the  straight/rectangle  model  was  viewed  first  by  18  students,  1 1 students  (61%)  indicated  that  the  curved/oval 
model  was  the  one  they  would  use  to  teach,  2 students  ( 1 1 %)  indicated  that  the  straight/rectangle  model  was  the  one  they 
would  use,  and  5 students  (28%)  indicated  that  they  had  no  preference  for  use. 


803 


589 


Model  Viewed  First 


Figure  8.  Number  of  students  selecting  each  model  as  “choice  to  use  to  teach” 


With  overall  data  strikingly  similar  to  that  of  Question  7, 20  students  (56%)  indicated  that  the  curved/oval  model  was  the 
model  they  would  use  to  teach  instructional  design  process,  7 students  (19%)  indicated  that  the  straight/rectangular  model 
was  their  choice,  and  9 (25%)  students  responded  that  they  did  not  have  a preference  of  model  to  use  to  teach  the  ID 
process  to  a colleague.  Additional  data  indicate  that,  though  students  strongly  tended  to  select  the  same  model  as  the 
“best”  one  and  their  “choice  to  use  to  teach/1  this  was  not  always  the  case. 

Data  displayed  in  Figures  9 and  10  also  provide  insight  into  the  students*  perceptions  of  the  “best”  and  “choose 
to  use  to  teach”  models  as  they  provide  verbal  explanations  of  their  decisions.  Again,  the  order  of  presentation  did  not 
appear  to  affect  the  terms  chosen  to  describe  student  reasoning  behind  their  preferences.  As  depicted  in  Figure  9, 
regardless  of  whether  it  was  presented  first  or  second,  if  the  curved/oval  model  was  preferred,  the  most  common  reasons 
included  that  it  was  more  flexible,  more  modifiable,  and  more  adaptable.  Regardless  of  whether  the  straight/rectangle 
model  was  viewed  first  or  second,  it  was  described  as  more  organized,  easier  to  follow,  and  easier  to  understand.  No 
comments  were  made  if  students  did  not  perceive  one  model  as  better  than  the  other.  In  a similar  manner,  results 
displayed  in  Figure  10  indicate  that  the  order  of  presentation  did  not  appear  to  affect  students*  choice  of  words  generated  to 
describe  their  preference  of  one  model  or  the  other  that  they  would  chose  to  use  when  teaching  someone  about  ID. 
Whether  presented  first  or  second,  the  curved/oval  model  was  generally  perceived  as  malleable,  not  as  fixed,  more  flexible, 
easier  to  understand,  and  more  flowing.  Again  regardless  of  presentation  order,  the  straight/rectangle  model  was  perceived 
as  more  organized,  more  clear,  more  structured,  easier  to  understand,  and  easier  to  follow.  Students  who  selected  no  model 
preference  indicated  that  they  would  integrate  the  use  of  both  models  but  did  not  supply  additional  words  to  describe  the 
models. 
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Model  Viewed  First 
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• More  flexible 

• More  circular — how  I see  the  process 

• Easy  to  modify 

• Easier  to  follow 

s Not  as  fixed 

• Flowed  smoothly 

o 

• Not  so  strict 

• Flexibility  in  design 

• More  like  a tool 

• More  pliable  instead  of  fixed 

• Not  set  in  stone 

• More  modifiable 

• Not  so  confusing 

• More  adaptable 

• More  flexible 

• More  straightforward 

• More  like  circuits  in  electricity 

• More  organization 

• More  organized 

• More  direct 

• Don't  like  curved  lines 

□ 

• Easier  to  understand 

• Easier  to  follow 

• More  structured 

• Easier  to  visualize  what  happens 
•More  easily  changed  & manipulated 

• More  logical 

• More  specific 

Neither 

No  Com 

jnents 

Figure  9.  Reasons  underlying  preference  for  “best”  model 


Discussion  and  Recommendations 

Regarding  Question  1,  data  indicate  that  preservice  teachers  changed  their  responses  between  the  pretest  and 
posttest  assessing  their  perceptions  of  the  flexibility,  organization,  and  value  of  the  ID  process.  Students'  perceived  the 
ID  process  as  more  flexible,  more  organized,  and  more  valuable/useful  after  the  instructional  intervention.  Student 
indications  that  ID  was  perceived  as  sensible  and  valuable  comfort  the  researchers,  who  believe  that  the  ID  process  indeed 
is  worthwhile  to  teach  to  these  preservice  teachers. 

Questions  2,  3,  and  4 may  be  affected  by  the  order  in  which  students  viewed  the  model,  but  evidence  that  they 
perceived  the  straight/rectangle  model  to  be  more  organized  was  highly  significant  (.01  level).  Students  who  viewed  the 
curved/oval  model  second  responded  significantly  (.05  level)  that  it  was  the  more  flexible  model.  This  perception  of 
greater  flexibility  is  also  supported  by  responses  to  the  open-ended  questions.  Conversations  with  Jerrold  Kemp 
(personal  communication,  February  17,  1996)  provide  additional  insight  into  this  observation.  Kemp  suggests  that 
straight/rectangle  models  may  provide  novice  instructional  designers  with  valuable  structure  and  organization  while  more 
mature  instructional  designers  appreciate  the  flexibility  represented  in  curved/oval  models.  This  is  an  avenue  for  further 
investigation. 
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Model  Viewed  First 
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• More  malleable 

• Not  as  fixed 

• Reflects  flexibility 

• More  laid  back 

• Explains  what  you  want 

• More  changeable 

• More  practical 

• Not  so  set 

• Flexibility  in  design 

• More  pliable 

• Less  fixed 

• Easier  to  understand 

• Flows  better 

• More  inviting 

• Less  confusing 

• More  personal  preference 

• More  organized 

• More  specific 

| • Don’t  like  curved  lines 

• Circut  could  be  modified 

1 • Clear 

• Specific 

• Clearer 

• More  professional 

• Straightforward 

• More  structured 

• Order  of  events 

• Explains  process  better 

• Less  confusing 

• Better  understood 

• More  organized 

• Easier  to  follow 

rj  No  Con 

iments 

Figure  9.  Reasons  underlying  preference  for  “choice  to  use  to  teach”  model 


In  considering  Question  5,  open-ended  responses  indicate  that  the  demographic  and  experiential  characteristics  of 
the  students  that  were  selected  for  this  study  may  not  be  as  influential  in  determining  student  preferences  as  other  factors 
not  yet  investigated.  Among  some  of  the  additional  characteristics  possibly  influencing  preservice  teachers'  perception  of 
the  ID  process  itself  and  the  way  in  which  it  is  visually  depicted  include  individual  learning  style  preferences;  cognitive 
style  factors  such  as  locus  of  control  and  field  dependence/independence;  educational  background  such  as  electrical 
engineering  training  or  experience  in  reading  other  types  of  flowcharts;  individual  preference  for  structure  or  flexibility; 
and  personal  visual  appeal.  In  conjunction  with  data  collected  to  address  Question  6,  this  observation  is  supported  by  the 
similarity  between  the  words  students  consistently  generated  to  describe  both  the  curved/oval  model  and  the 
straight/rectangle  model  and  the  ID  process  itself.  Apparently,  student  characteristics  other  than  those  identified  have  a 
greater  impact  on  their  perceptions  of  the  ID  process  and  the  visual  representation  of  the  model,  and  this  is  an  area  for 
continued  investigation. 

With  regard  to  Questions  7 and  8,  a majority  of  students  (56%  in  each  case)  indicated  a preference  for  the 
curved/oval  model  as  the  “best  model  to  represent  the  ID  process”  and  as  the  “choice  of  model  to  use  to  teach  the  ID 
process.”  However,  data  do  not  indicate  why  this  preference  exists,  particularly  when  a statistically  significant  number  of 
students  indicated  that  the  straight/rectangle  model  was  more  organized.  This  is  another  area  in  which  further 
investigation  is  being  conducted. 

The  importance  of  the  visual  display  of  ID  models  for  professionals  teaching  the  instructional  design  process  is 
summarized  by  two  very  different  quotations  from  two  sophomore  students,  both  of  whom  had  no  preference  for  a “best” 
or  “choice  to  use  to  teach”  model.  One  student  lamented:  “I  understand  the  presentation  that  you  gave  and  it  was  very 
helpful . . . but  I don't  get  these  dang  models!”  This  comment  underscores  the  importance  of  presenting  a visual 
representation  of  the  ID  process  that  facilitates — not  hinders — students’  acquisition  of  the  principles  underlying  the 
instructional  design  process.  Numerous  comments  also  could  be  cited  to  support  the  observation  of  students'  preferences 
of  models  based  on  their  own  personal  preferences  for  structure  or  flexibility.  However,  one  very  astute  student 
concluded:  “The  only  difference  is  the  lines  as  far  as  I could  see.  To  me  lines  that  arc  curved  or  lines  that  are  straight 
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still  point  to  the  same  thing.  . . . Curved  lines  may  indicate  flexibility  & straight  lines  more  structure  & less  variance. 

I look  at  the  design  as  both  flexible  & structured.”  Since  this  is  one  of  the  key  messages  regarding  the  instructional 
design  process  that  the  researchers  attempt  to  convey  to  preservice  teachers,  perhaps  preservice  teachers  should  be 
introduced  to  both  curved/oval  and  straight/rectangle  visualizations  of  the  ID  process.  This  study  has  generated  more 
questions  than  it  has  answered,  and  the  question  of  the  impact  and  importance  of  the  visual  display  of  an  ID  model  on 
student  perceptions  of  the  process  itself  remains  a viable  area  for  additional  research. 
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Instructional  Technology  is  a field  that  has  grown  from  two  separate  knowledge  bases  and  two  related  areas  of 
practical  concern.  While  its  original  roots  were  in  the  study  and  construction  of  visual  aids  as  teaching  devices,  the 
second  major  line  of  intellectual  heritage  emanated  from  instructional  psychology,  and  provides  the  bases  for  many 
principles  of  instructional  design  theory  and  practice.  Gagne  has  been  a central  figure  in  this  infusion  of  psychology  into 
the  field,  and  indeed  in  the  “creation”  of  the  domain  of  instructional  design.  Today,  the  bulk  of  the  research  and  theory  in 
Instructional  Technology  is  concentrated  in  the  design  domain  with  less  research  emphasis  place  upon  utilization, 
management,  evaluation,  and  even  the  important  domain  of  development  (Seels  & Richey,  1994). 

Instructional  design  can  be  seen  in  terms  of  either  macro-design  procedures  which  provide  overall  direction  to  a 
design  project  (typically  using  instructional  systems  design  principles)  or  micro-design  that  involves  the  design  of 
lessons  and  instructional  strategies  which  constitute  those  lessons.  Gagne’s  contributions  have  been  primarily  in  the 
development  of  micro-design  principles  and  procedures. 

Gagne’s  orientation  to  design  now  serves  as  the  crux  of  most  micro-design  models,  such  as  in  Instructional 
Transaction  Theory  (Merrill,  Li,  & Jones,  1991),  Elaboration  Theory  (Reigeluth  and  Stein,  1983),  or  the  ARCS  Model 
of  Motivation  Design  (Keller,  1987).  In  each  of  these  models,  learning  is  fundamentally  viewed  as  an  internal  process 
that  is: 

• dependent  upon  past  learning;  and 

• stimulated  and  controlled  by  external  events; 

• expedited  by  instruction  which  varies  depending  upon  the  nature  of  the  desired  outcome;  and 

• precipitated  by  the  use  of  sequenced  instructional  strategies  that  provide  motivation,  direction, 
guided  practice,  feedback,  and  reinforcement. 

These  ideas  emanate  substantially  from  the  work  of  Robert  Gagne. 

The  principles  promoted  by  Gagne  provide  not  only  a theoretical  orientation  to  an  instructional  design  project, 
but  also  have  prompted  a number  of  design  conventions  and  techniques.  His  ideas  are  now  reflected  in  a variety  of  work 
environments,  including  corporate  training,  the  military,  the  health  care  industry,  as  well  as  educational  settings.  In 
addition,  his  design  principles  have  been  integrated  into  delivery  systems  of  all  types. 

The  key  question  to  be  examined  in  this  paper,  however,  is  “To  what  extent  will  Gagne’s  theories  continue  to 
influence  the  field  as  design  research  expands  and  as  design  practice  changes  in  response  to  new  demands  and  pressures?”. 
While  Gagne’s  dominance  has  been  assumed  in  the  past,  this  question  is  not  unrealistic  in  the  current  climate  of  growing 
alternative  perspectives  of  both  research  and  design  processes.  Today’s  intellectual  climate  is  subject  to  many  pressures 
from  new  theoretical  orientations,  as  well  as  from  on-the-job  demands  for  additional  efficiency  and  effectiveness.  These 
changing  pressures  and  ideological  influences  may  also  control  the  impact  of  the  Gagne  design  orientation. 

Nonetheless,  Gagne’s  influence  has  been  so  pervasive  that  it  is  easy  to  find  traces  of  Gagne  theory  even  in  the 
most  divergent  design  orientations.  The  aim  of  this  paper  is  not  so  much  to  further  identify  Gagne’s  imprint  on  our  field 
in  emerging  design  theory  and  practice,  but  rather  to  analyze  these  trends  in  an  effort  to  predict  the  stability  and 
continuing  relevance  of  his  theory. 

The  Continuing  Domination  of  Gagne  Doctrines  in  Design  Theory 

Previously  unvoiced  challenges  to  traditional  principles  of  instructional  design  theory  are  now  surfacing  in  the 
field.  To  a great  extent,  these  challenges  stem  from  criticism  of  our  heavy  reliance  upon  general  systems  theory  and 
conventional  learning  theory.  New  learning  and  instructional  theory,  especially  those  positions  concerning  the  role  of 
the  learning  task,  the  impact  of  the  learner  and  learner  control,  and  increased  concern  with  the  need  for  transfer  of  training, 
is  closely  related  to  Gagne’s  work.  These  innovations  are  presented  within  the  context  of  constructivism,  situated 
learning,  and  an  even  more  pronounced  emphasis  on  cognitive  psychology  principles.  As  a whole,  they  raise  the 
possibility  of  fairly  profound  changes  in  design  practice. 


The  Emerging  Tension  Between  Learner-Oriented  and  Content-Oriented  Instruction 

Trends  in  Learner-Centered  Instruction.  Instructional  design  procedures  are  guided  by  both  the  nature  of  the 
learning  task  and  the  nature  of  the  learner.  With  respect  to  the  learning  task,  Gagne’s  work  leads  to  an  analysis  of  the 
content  so  that  one  can  not  only  determine  the  sequence  of  a lesson,  but  also  diagnose  the  prerequisite  skills  of  the  learner 
(Gagne,  1962,  1968a;  1968b).  Instructional  strategies  are  also  contingent  upon  the  learning  task,  since  they  vary 
depending  upon  the  type  of  task  being  addressed.  For  example,  problem  solving  tasks  are  taught  differently  from  concept 
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formation  tasks.  While  learner  characteristics  are  clearly  important,  traditionally  instructional  design  procedures  have 
been  controlled  more  by  the  material  to  be  taught  than  by  the  persons  receiving  the  instruction.  This  position  is  totally 
compatible  with  the  objectives-oriented  stance  of  instructional  systems  design  models.  However,  to  many  it  is  also  an 
outdated  vestige  of  behavioral  learning  theory. 

Currently,  there  are  trends  toward  a change  in  this  stance  with  much  of  the  new  theoretical  thinking  focusing 
more  centrally  on  the  learner,  with  content  taking  a somewhat  secondary  role  (Richey,  1993).  This  is  most  evident  in 
constructivist  theory  that  posits  that  meaning  and  reality  are  functions  of  individual  interpretation  (Jonassen,  1991; 
Lebow,  1993),  and  learning  itself  is  a process  ... 

...  in  which  the  learner  is  building  an  internal  representation  of  knowledge. ...  This  representation  is  constantly 
open  to  change,  its  structure  and  linkages  forming  the  foundation  to  which  other  knowledge  structures  are 
appended.  Learning  is  an  active  process  in  which  meaning  is  developed  on  the  basis  of  experience  (Bednar, 
Cunningham,  Duffy,  & Perry,  1992,  p.  21). 


Central  to  constructivism  in  its  most  extreme  form  is  the  rejection  of  the  notion  of  an  objective  reality  and  the 
role  of  external  events  (i.e.,  external  conditions  of  learning)  as  ways  of  promoting  a common  reality  for  a group  of 
people  (Jonassen,  1991).  In  other  words,  the  emphasis  on  an  individual’s  internal  processing  of  information  is 
completely  dominant.  The  learner  and  learning  processes  control,  and  even  shape,  the  learning  task. 

In  spite  of  the  growing  popularity  of  constructivist  principles,  many  designers  are  uncomfortable  with  extreme 
constructivist  positions.  Dick’s  (1992)  comments  in  this  regard  are  noteworthy.  He  observes  that  educational 
interventions  that  are  truly  constructivist  must  necessarily  provide  the  learner  with  almost  total  control  of  the 
instructional  process,  including  the  selection  of  objectives  and  learning  activities.  This  is  less  because  of  an  adherence  to 
laissez-faire  philosophy  than  it  is  to  a belief  that  pre-designed  instruction  is  meaningless  since  two  individuals  will 
seldom  have  the  same-interpretation  or  understanding  of  a particular  event.  This  position  minimizes  the  role  of  structured 
instructional  activities,  in  favor  of  a more  tutorial  model  of  instructional  facilitation.  Merrill  (1992)  also  argues  against 
extreme  constructivism  by  pointing  out  that  while  learners  develop  their  own  internal  mental  models  in  most 
instructional  situations,  they  nonetheless  do  respond  to  and  interact  with  abstractions  and  mental  models  of  others.  There 
is  a functional,  objective  core  of  knowledge  that  can  easily  co-exist  with  individual  interpretations  and  opinions  of  the 
knowledge. 

Questions  regarding  the  dominance  of  content  are  not  confined,  however,  to  constructivists.  Many  involved  in 
design  and  development  of  instruction  using  the  new  and  emerging  technologies  have  also  come  to  question  traditional 
practice.  Hannafin  (1992)  argues  that  traditional  instructional  design  theory  and  models  are  most  effective  with  “highly 
prescribed,  objective  outcomes  and  the  organization  of  to-be-leamed  lesson  content,  not  the  largely  unique  and  individual 
organization  of  knowledge"  (p.  50).  Instruction  controlled  by  these  pre-defined  objectives  then  tends  to  be  seen  as  only 
the  transmission  of  knowledge,  rather  than  the  facilitation  of  learning.  Kember  and  Murphy  (1990)  voice  concerns  that 
the  product  of  such  instruction  is  only  surface,  rather  than  deep  learning.  It  is  learning  that  is  less  likely  to  be  transferred 
and  used  by  the  learner  in  new  situations. 

It  is  predictable  that  constructivists  and  technology  developers  would  find  common  ground  in  these  arguments. 
The  new  technologies  can  facilitate  levels  of  learner  control  previously  unattainable.  The  technologies  can  give  learners 
instant  access  to  information,  and  the  ability  to  link  information.  The  technologies  allow  totally  adaptable,  interactive, 
and  less  structured  designs  and  learning  environments  than  were  typical  using  traditional  design  orientations  (Hannafin, 
1992).  Here  the  content  becomes  part  of  the  learning  environment,  but  the  learner  is  more  dominant. 

Learner-Centered  Design  and  Gagne.  The  learner-centered  advocates,  even  those  who  espouse  a more  extreme 
position,  do  not  have  theoretical  bases  that  are  totally  incompatible  with  Gagne’s  theory.  He  stresses  the  importance  of 
prior  knowledge  and  experience,  self-regulated  learning  strategies,  and  learner  motivation.  However,  there  is  a different 
perspective  in  the  Gagne  orientation.  While  Gagne  uses  learner  characteristics  as  one  basis  of  strategy  selection,  the 
pertinent  learner  considerations  tend  to  relate  to  the  cognitive  processing  of  information  - the  nature  and  capacity  of  the 
learner’s  memory  storage  and  processing  skills,  qualities  that  affect  sensory  perception,  and  attitudes  that  directly  impinge 
on  learning.  These  are  factors  which  shape  one’s  ability  and  motivation  to  achieve  a given  type  of  learning  outcome. 
They  define  the  learners’  cognitive  capabilities,  prerequisite  content  knowledge,  and  interest  in  a particular  topic,  and 
become  central  to  the  design  of  those  external  conditions  that  promote  learning.  Learner  characteristics,  however,  are 
critical  to  Gagne’s  theory  primarily  in  the  extent  to  which  they  are  related  to  pre-defined  learning  objectives. 

Learner  involvement  (as  opposed  to  learner  control)  is  also  a critical  aspect  of  Gagne’s  work,  but  learners’ 
participation  in  the  instructional  process  entails  more  than  simply  being  engaged  in  a series  of  activities,  the  external 
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performance  aspect  of  instruction.  Participation  and  activity  also  refer  to  internal  involvement  in  the  perception,  storage, 
and  retrieval  of  information.  This  is  the  core  of  Gagne’s  cognitive  orientation,  and  the  Events  of  Instruction  are  designed 
to  promote  internal,  as  well  as  external  activities. 

Current  advocates  of  learner-centered  instruction  present  arguments  that  are  multi-faceted,  including  debates  on  at 
least  two  aspects  of  the  problem: 

• who  controls  the  instructional  process,  and  what  is  the  nature  of  such  control?  and 

• which  learner  characteristics  influence  the  selection  and  design  of  instructional  strategies  and  how 
they  should  be  addressed? 

With  respect  to  the  first  question,  Gagne  seems  to  suggest  that  the  designer  (or  the  instructor)  has  fundamental  control  of 
instructional  processes  that  are  external  events,  even  though  individuals  always  control  their  own  learning  processes 
which  are  internal  events.  Superior  design  of  instruction  can  facilitate  learning  efficiency,  instructional  effectiveness, 
transfer  of  training,  and  interest.  In  these  respects  then,  the  designer  also  exercises  a certain  amount  of  control  by 
structuring  the  external  conditions  according  to  research-based  principles  in  a manner  that  will  facilitate  internal  learning 
and  information  processing.  Control  of  the  teaching/leaming  process  then  is  shared  by  learners  and  designer/instructors. 

Although  seldom  framed  in  these  terms,  the  issues  surrounding  control  of  instruction  seem  to  have  much  to  do 
with  the  type  and  extent  of  individualization  that  is  desirable  in  a teaching/leaming  environment.  Individualized 
instruction  has  always  been  valued  by  instructional  technologists,  in  spite  of  the  many  interpretations  of  the  term. 
Fundamentally,  individualized  instruction  involves  varying  the  teaching/leaming  procedures  for  each  student.  These 
variations  occur  by  making  different  instructional  decisions  for  different  students.  These  decisions  include: 

• what  and  how  much  should  one  learn? 

• when  and  where  should  one  learn? 

• what  resources  should  one  use  to  learn? 

• how  does  one  know  when  learning  has  occurred  or  when  it  has  not? 

The  extent  of  individualization  depends  upon  the  number  of  decisions  made  for  individuals  as  opposed  for  the 
class  as  a whole,  and  the  extent  to  which  learners  assume  control  of  the  decisions  regarding  their  own  instruction.  For 
example,  an  individualized  setting  may  only  involve  self-pacing  of  instruction  by  the  students  with  the  content, 
materials,  and  testing  procedures  prescribed  by  the  instructor.  On  the  other  hand,  a program  may  be  totally  individualized 
with  learners  making  all  of  their  own  decisions,  and  instructors  serving  as  facilitators.  Technologically-based  delivery 
systems  clearly  expedite  learner  control.  For  example,  most  hypertext  environments  at  the  least  allow  students  to  control 
content  selection,  sequencing,  and  pacing. 

Constructivists  and  many  of  those  involved  in  using  the  new  technologies  to  their  fullest  advantage,  tend  to 
advocate  more  total  individualization.  Such  instruction  is  not  necessarily  incompatible  with  Gagne’s  principles  of 
learning.  Moreover,  the  compatibility  is  not  dependent  upon  whether  there  is  a structured  or  flexible  approach  to  the 
management  of  its  delivery.  Learners  can  assume  a major  part  of  the  control  of  the  instruction,  and  the  process  can  still 
be  perfectly  consistent  with  Gagne’s  theory.  The  critical  factor  seems  to  be  whether  the  design  of  this  instruction  has 
been  grounded  in  an  analysis  of  the  subject  matter  and  the  learner  prerequisites,  not  on  whether  students  are  involved  in 
collaborative,  active  learning  of  highly  relevant  content. 

In  some  respects,  the  second  aspect  of  the  learner-centered  issue  is  more  interesting  — the  most  critical  learner 
characteristics.  Instructional  design  rooted  in  a content  emphasis,  as  opposed  to  a learner  emphasis,  tends  to  highlight 
learner  traits  that  are  related  in  some  way  to  the  subject  matter  of  the  lesson,  including: 

• prerequisite  skills; 

• background  experiences  which  enhance  prerequisites  and/or  interest  in  the  topic  of  the  lesson;  or 

• the  learner’s  proficiency  in  those  cognitive  strategies  required  to  master  the  content. 

Designers  are  now  considering  other  learner  characteristics  as  well.  For  example,  Richey  (1992)  has  shown  the 

direct  impact  on  learning  of  other  learner  characteristics  that  are  not  content-related.  With  respect  to  adult  learning  in 
employee  training  environments,  pertinent  factors  include  learner  attitudes  toward  the  instructional  delivery  system  and 
the  organization  delivering  the  training,  the  previous  training  experiences  of  learners,  and  their  work  experience.  These 
learner  attitudes  and  background  experiences  seem  to  predict  not  only  the  extent  to  which  objectives  are  achieved  during 
training,  but  also  the  extent  of  transfer.  Other  characteristics  currently  being  studied  include  feminist  thinking  (Gilligan, 
1982;  Canada  & Brusca,  1991)  and  other  aspects  of  a person’s  cultural  background. 

It  is  possible  for  one  to  argue  that  these  characteristics  shape  the  cognitive  strategies  that  a learner  uses  to 
address  a particular  piece  of  instructional  content  and,  as  such,  are  still  within  the  Gagne  tradition.  Nonetheless,  they  do 
represent  a line  of  thinking  which,  even  though  logically  connected  to  Gagne’s  previous  work,  is  suggesting  new  design 
procedures  and  emphases.  It  is  one,  however,  which  is  consistent  with  Gagne’s  past  thinking. 
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The  Emerging  Role  of  Context  in  Instructional  Design  Theory 

Trends  in  Context-Centered  Instruction.  Another  area  of  current  theoretical  expansion  concerns  the  impact  of 
context  upon  the  teaching/learning  process.  Of  interest,  is  not  only  the  immediate  teaching  context,  but  also  the  pre- 
instructional  and  post-instructional  environments  in  which  learners  live  and  work  (Richey,  1993;  Richey  & Tessmer, 

1995). 

This  trend  of  looking  to  contextual  variables  as  predictors  of  learning  effectiveness  has  emerged  with  the 
concurrent  influence  of  performance  technology,  the  quality  movement,  situated  learning,  systemic  design,  and  once 
again,  constructivism.  The  commonalty  among  these  divergent  theories  and  movements  is  an  interest  in  “meaningful” 
instruction,  meaningful  to  the  learner  and  meaningful  to  the  society  that  expects  to  be  improved  as  a result  of  an  educated 
populace.  In  the  past,  such  “meaning”  has  had  important  implications  for  the  transfer  of  training  from  educational 
environments  to  real-life  behavior.  Today,  it  also  has  implications  for  organizational  development  and  quality 
improvement. 

Most  instructional  design  procedures  and  principles  are  typically  seen  as  being  applicable  to  all  settings.  In 
spite  of  this,  new  instructional  systems  design  procedural  models  are  frequently  developed  in  an  effort  to  respond  to  the 
seemingly  unique  aspects  of  a given  situation.  This  emphasis  on  situation-specific  procedures  is  complemented  by  the 
ever  increasing  demands  that  education  and  training  programs  serve  as  quality  improvement  vehicles  and  solve  specific 
organizational  problems.  Contextualization  also  reflects  efforts  to  create  motivating,  relevant  instruction.  While  this 
latter  goal  is  not  new,  the  pressures  for  intrinsically  relevant  instruction  are  increasing  with  the  new  emphases  on  adult 
education  and  training  and  the  expectations  of  children  and  adolescents  reared  on  action-oriented  television.  Finally,  these 
events  coincide  with  situated  learning  and  constructivist  emphases  on  “anchoring”  instructional  activities  into  meaningful 
contexts  as  a means  for  promoting  long-term  retention,  understanding,  and  transfer  of  training.  The  issue  is  then  an 
outgrowth  of  societal  changes,  as  well  as  new  theoretical  biases. 

The  emphasis  on  context  and  environment  is  not  unrelated  to  the  learner-centered  design  thrust,  since  context  is 
typically  a matter  of  perceptions  made  by  learners  in  light  of  their  background  experiences.  Moreover,  context  emphases 
also  tend  to  expand  the  number  of  factors  addressed  by  designers,  sometimes  at  the  expense  of  instructional  content 
considerations. 

Contextualization  is  typically  achieved  not  only  through  the  topics  of  instruction,  but  also  through  the  selection 
of  examples  and  the  nature  of  the  practice  exercises.  Topics  can  be  those  that  are  currently  issues  in  a particular  setting. 
Examples  can  be  drawn  from  the  social  or  work  culture  of  the  students.  Practice  can  be  provided  using  what  Brown, 
Collins,  & Duguid  (1989)  would  call  authentic  activities.  Authenticism  involves  “ordinary  practices  of  the  culture”  (p. 
34),  as  opposed  to  hybrid  activities  that  are  more  reflective  of  the  education  and  training  culture  rather  than  the  “real 
world”.  Decontextualized  environments,  therefore,  are  not  only  created  through  the  use  of  verbal  abstractions,  but  also 
through  the  use  of  examples  and  practice  activities  that  are  not  reflective  of  the  daily  situations  encountered  by  the 
learners.  One  can  also  create  context-rich  instruction  by  using  problems,  examples,  and  practice  activities  involving 
multiple  contexts.  In  this  way,  instruction  seems  realistic,  even  though  it  is  not  “anchored”  in  a given  context.  This  is 
not  the  typical  approach,  however,  in  many  of  the  newer  approaches  to  context  in  instruction. 

Current  emphases  on  context  have  the  potential  of  changing  design  procedures  by  not  only  expanding  the  needs 
assessment,  evaluation,  and  systems  maintenance  phases,  but  also  by  altering  the  nature  of  the  instructional  strategies 
themselves.  The  ultimate  goal  is  instruction  that  is  less  abstract,  more  applied,  and  more  responsive  to  external  realities 
than  had  previously  been  the  case. 

Context-Centered  Design  and  Gagne.  Streibel  (1991)  summarizes  the  fundamental  difference,  in  his  opinion, 
between  Gagne's  theory  of  instruction  and  that  of  situated  learning  with  respect  to  contextual  issues.  He  sees 
environmental  factors  in  the  Gagne  tradition  as  playing  the  role  of  triggering  stimuli  in  a teaching/learning  situation, 
rather  than  serving  as  causes  of  behavior.  While  this  characterization  may  be  debatable,  the  point  is  well  made  that 
context  is  not  as  central  in  the  Gagne  theory  as  it  is  in  many  current  orientations,  and  the  question  at  hand  involves  the 
extent  of  this  deviation.  This  issue  can  be  analyzed  in  terms  of  the  implications  of  context  for  transfer  of  training  as  well 
as  long-term  retention  - elements  that  are  not  unrelated  and  need  to  be  considered  together. 

Transfer  of  training,  from  Gagne's  perspective,  is  a function  of  the  extent  to  which  a learner  has: 

• the  required  prerequisite  knowledge  and  skills; 

• the  ability  to  recall  prior  learning;  and 

• developed  those  cognitive  strategies  appropriate  for  the  task. 

The  first  is  a function  of  content  and  background,  rather  than  contextual  elements  of  the  instruction.  The  latter 
two  elei  icnts,  however,  are  impacted  by  context.  The  ability  to  recall  needed  prior  learning  is  a function,  in  part,  of 
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whether  the  material  to  be  recalled  was  originally  presented  within  a meaningful  contextual  framework.  If  so,  it  is  far 
easier  to  recall.  Moreover,  the  contextual  anchoring  of  past  instruction  in  a variety  of  novel  proble  i solving  tasks  not 
only  enhances  meaning,  but  also  develops  cognitive  strategies  used  in  problem  solving  and  transfer  of  training.  Gagne, 
therefore,  tends  to  advocate  context-rich  instruction  by  systematically  using  alternative  contexts  for  practice,  rather  than 
emphasizing  the  dimensions  of  only  one  environment. 

What  is  more  likely  to  strengthen  transfer,  generalization  or  context-embedded  instruction?  Clark  and  Voogel 
(1985)  conclude  that  “the  extent  of  transfer  is  determined,  in  part,  by  the  amount  of  decontextualization  achieved  during 
instruction”  (p.  1 19),  but  that  the  issue  is  also  dependent  upon  the  nature  of  the  learning  task  and  the  type  of  transfer 
anticipated.  Procedural  knowledge  is  more  conducive  to  near  transfer  (i.e.  transfer  of  skills  to  situations  which  are  similar 
to  those  in  which  the  instruction  occurred),  while  concepts  and  principles  are  more  appropriate  for  far,  or  more 
generalized,  transfer  situations.  Moreover,  Clark  and  Voogel  (1985)  assert  that  the  two  types  of  transfer  are  not 
compatible;  one  is  typically  emphasized  at  the  expense  of  the  other,  even  though  all  transfer  is  highly  dependent  upon 
learner  abilities.  In  this  vein,  Gagne  would  likely  assert  that  even  though  putting  instruction  into  a meaningful  context 
is  important,  instruction  that  is  dominated  by  examples  from  real-life  situations  is  not  necessarily  in  the  best  position  to 
promote  the  process  of  far  transfer.  This  point  is  one  important  part  of  the  discussion  of  differences  between  Gagne  and 
the  advocates  of  highly  context-centered  instruction. 

Perkins  and  Salomon  (1989)  in  essence  have  discussed  the  same  issue,  but  in  terms  of  the  dichotomy  between 
the  roles  of  general  strategic  knowledge  (i.e.  decontexualized)  and  specialized  domain  knowledge  (i.e.  contextualized)  as 
predictors  of  effective  problem  solving.  They  conclude  that  transfer  is  a highly  specific  phenomenon  and  while  all 
specific  applications  need  to  consider  contextual  factors,  there  is  a need  to  have  an  “intimate  intermingling  of  generality 
and  context-specificity  in  instruction”  (p.  24).  This  seems  not  so  very  different  from  Gagne’s  position  of  embedding 
context  within  the  instructional  strategies,  even  though  the  ultimate  goal  is  to  facilitate  far  transfer. 

There  is  a second  aspect  of  Gagne’s  work  and  orientation  that  needs  to  be  considered  when  discussing  the  role 
of  context  in  promoting  both  transfer  of  training  and  long-term  retention  - namely,  the  role  of  the  enterprise  schema 
(Gagne  & Merrill,  1990).  An  enterprise  is  a complex  purposive  performance  involving  multiple,  related  instructional 
goals.  It  is  a higher  level  goal  than  is  frequently  used  in  many  education  and  training  programs.  An  enterprise  is 
represented  in  one’s  memory  by  a schema  that  relates  these  larger  goals  (typically  presented  as  a realistic  application  task) 
to  their  prerequisite  skills  and  knowledge.  The  schema  is  a mental  model  that  serves  as  the  basis  for  both  retention  and 
retrieval,  as  well  as  transfer. 

The  emphasis  on  integrated  instructional  goals  corresponds  with  an  emphasis  on  purposive,  relevant  instruction. 
While  such  instruction  is  designed  with  transfer  in  mind,  it  may  not  be  authentic  in  the  same  sense  promoted  by 
advocates  of  situated  cognition  that  seems  to  view  learning  as  more  of  an  enculturation  process.  The  notions  of 
integrated  goals  and  enterprise  schema  tend  to  relate  more  to  generalized  transfer  and  a de-emphasis  of  declarative  and 
procedural  learning  as  an  end  unto  itself.  While  Gagne  would  undoubtedly  use  contextualized  examples  and  practice 
activities,  it  is  unlikely  that  he  would  advocate  always  rooting  instruction  in  a single , even  though  relevant,  context. 

The  most  useful  enterprise  schema  is  somewhat  generic,  applicable  to  a variety  of  specific  enterprises  in  which  one 
might  become  engaged.  Of  course,  much  instruction,  especially  that  of  a training  nature,  is  oriented  only  toward  specific 
performance-oriented  objectives  that  are  more  conducive  to  near  transfer,  and  these  situations  often  demand  strategies 
which  utilize  a given  context. 

In  summary,  Gagne’s  orientation  to  context  is  not  totally  incompatible  with  current  thinking  insofar  as  it  stems 
from  a cognitive  orientation.  Gagne  continues  to  strive  for  instruction  that  primarily  addresses  higher  levels  of  learning 
and  aspires  to  far  reaching  transfer  as  opposed  to  specific  applications  of  content.  Effective  instruction  is  relevant  to 
learners’  needs  as  well  as  being  appropriate  to  their  skill  levels,  but  probably  shows  application  in  a variety  of  contexts 
rather  than  being  “anchored”  in  only  one  environment.  While  procedural  knowledge  is  apt  to  be  related  to  more  specific 
uses,  the  goal,  nonetheless,  is  that  of  using  such  knowledge  in  combination  with  other  skills  and  knowledge  for  creative 
problem  solving.  Ultimately,  Gagne’s  design  theory  is  generic  in  nature.  It  is  theory  that  is  applicable  to  all  contexts, 
all  types  of  content,  and  all  types  of  learners. 

The  Stability  of  the  Gagne  Orientation  to  Theory 

A clear  trend  in  design  theory  over  the  past  fifty  years  has  been  its  continual  expansion.  There  is  more  research. 
There  is  more  theory  construction.  Just  as  Gagne  responded  to  those  issues  that  were  critical  during  his  most  productive 
years,  today’s  scholars  are  responding  to  a new  set  of  concerns.  While  it  is  evident  that  the  new  theorizing  is  at  times 
charting  new  waters,  for  the  most  part  new  theory  is  not  antithetical  to  the  old  and,  it  continues  to  build  upon  Gagne’s 
foundational  work.  It  is  likely  that  Gagne’s  primary  positions  will  remain  current  to  the  extent  that: 
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• cognitive  learning  principles  continue  to  be  accepted; 

• design  continues  to  be  viewed  as  a generic  activity;  and 

• instructional  content  and  strategies  continue  to  be  pre-specified  and  analyzed. 

There  are  now  alternatives  to  each  of  these  perspectives  that  present  radically  new  design  orientations;  although,  they 
remain  on  the  fringe  of  acceptability.  Their  acceptance  as  more  mainstream  theory  could  modify  Gagne’s  impact  on 
future  theory  development.  However,  at  this  time,  such  a scenario  seems  unlikely. 

The  Continuing  Domination  of  Gagne  Doctrines  in  Design  Practice 

As  with  theory,  the  world  of  design  practice  is  also  undergoing  changes  that  were  previously  unanticipated. 

These  changes  are,  on  the  whole,  reactions  to  demands  for  increased  design  efficiency.  Such  concerns  are  especially 
reflected  in  current  efforts  to  enhance  the  traditional  instructional  systems  design  models  and  to  reduce  design  cycle  time. 

In  the  preface  to  the  first  edition  of  The  Conditions  of  Learning  Gagne  (1965)  indicated  that  the  impetus  for  this 
book  was  to  explain  “what  is  known  about  the  process  of  learning  that  can  be  put  to  use  in  designing  better  education” 

(p.  v).  This  reflects  the  fundamentally  practical  nature  of  instructional  design.  From  one  edition  to  the  next.  The 
Conditions  of  Learning  became  increasingly  more  practice  oriented,  and  the  final  edition  included  four  chapters  describing 
specific  design  and  analysis  procedures  and  techniques.  The  question  now  is  basically  the  same  as  was  posed  with  respect 
to  his  theoretical  contributions.  Will  Gagne’s  design  and  development  techniques  continue  to  provide  direction  for  the 
typical  practitioner? 

In  the  past  much  of  his  direction  for  practice  has  related  to  techniques  for  varying  designs  in  terms  of  the  type  of 
learning  task,  for  using  learning  hierarchies  as  a pre-design  content  analysis  tool,  and  for  using  the  events  of  instruction 
as  a guide  for  the  design  of  lessons  and  the  selection  of  instructional  strategies.  Clearly  these  tactics  are  bi-products  of 
Gagne  design  theory.  While  their  continued  use  by  the  typical  practitioner  is  dependent  upon  the  stability  of  this 
underlying  theory,  such  use  is  also  greatly  affected  by  the  realities  of  the  everyday  world  of  work. 

The  Continuing  Dominance  of  Conditions  and  Outcomes-Based  Design 

The  core  of  Gagne’s  contributions  to  instructional  design  relates  to  the  premise  that  learning  is  brought  about 
by  arranging  different  instructional  conditions  for  different  types  of  learning  tasks.  Gagne  has  identified  five  different 
domains  of  learning  outcomes,  and  has  suggested  varying  conditions  that  are  likely  to  lead  to  a learner  achieving  each  of 
these  types  of  goals  (Gagne  1965, 1972).  This  approach  is  foundational  to  most  instructional  design  models.  It  is 
difficult  in  today’s  climate  to  imagine  a field  of  instructional  design  without  such  an  orientation. 

While  there  is  currently  more-advocacy  of  alternative  design  positions  than  has  previously  been  the  case,  this 
principle  seems  to  be  essentially  unchallenged  by  practicing  designers.  It  is  not  that  other  instructional  foundations  have 
not  been  suggested,  such  as  the  developmental  level  of  the  learner,  or  the  use  of  reinforcement.  While  Gagne’s 
conditions  of  learning  recognize  the  role  of  such  elements,  the  basic  principle  remains  --  instruction  should  vary 
depending  upon  what  is  to  be  learned. 

The  only  major  dissension  with  respect  to  this  position  is  voiced  by  constructivists  who  do  not ... 

...  accept  the  assumption  that  types  of  learning  can  be  identified  independent  of  the  content  and  the  context  of 
learning.  Indeed,  from  a constructivist  viewpoint  it  is  not  possible  to  isolate  units  of  information  or  make  a 
priori  assumptions  of  how  the  information  will  be  used.  Facts  are  not  simply  facts  to  be  remembered  in 
isolation  (Bednar  et  al.,  1992,  p.  23). 

Gagne  (as  have  most  instructional  designers)  has  often  noted  the  futility  of  teaching  isolated  facts,  even  though 
he  would  nonetheless  argue  that  such  content  can  be  classified.  It  is  far  more  common  today  for  the  Gagne  position  to 
be  supported  with  respect  to  this  issue,  and  there  is  no  indication  that  his  basic  premise  will  not  remain  essentially  in 
tact.  Other  elements  of  design  practice  according  to  Gagne,  however,  are  to  some  extent  more  debatable,  even  though 
they  too  are  prevalent  among  practitioners. 

The  Continuing  Dominance  of  Pre-Design  Analysis 

Gagne’s  emphasis  on  pre-design  content  analysis  coincides  with  the  tenets  of  general  systems  theory.  The  use 
of  the  learning  hierarchy  tool  facilitates  such  analysis  as  well  as  the  identification  of  necessary  learner  prerequisites. 
Today  there  are  two  seemingly  opposing  trends.  The  first  is  to  expand  the  analysis  stage  to  accommodate  a wider  range 
of  design  variables  in  an  effort  to  promote  transfer  of  training  (Richey,  1995).  The  second  is  a recognition  that  many 
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expert  designers  use  other  methods  that  do  not  depend  upon  such  analysis  (Tripp,  1994).  Both  trends  are  occurring  in  the 
midst  of  extreme  pressure,  especially  in  the  world  of  business,  to  reduce  the  design  cycle  time. 

Before  predicting  the  continuing  influence  of  Gagne  with  respect  to  pre-design  analysis,  it  is  important  to  try  to 
do  justice  to  his  position.  With  respect  to  the  use  of  analysis  and  the  construction  of  learning  hierarchies,  Gagne  has 
cautioned  against  rigid  use  of  the  technique.  For  example,  he  recognizes  that  a learning  hierarchy  is  not  necessarily  the 
sequence  by  which  an  individual  learner  will  acquire  a particular  capability,  rather  it  is  the  most  probable  route  to  transfer 
of  training  for  most  people.  He  also  cautions  against  emphasizing  verbal  knowledge  in  a hierarchy  at  the  expense  of  the 
underlying  intellectual  skills. 

These  arguments  (made  over  twenty-five  years  ago)  may  anticipate,  at  least  in  part,  current  analysis  trends.  The 
expansion  of  the  analysis  phase  today  represents  not  only  the  increased  attention  being  given  to  learner  characteristics  and 
context,  but  also  a new  adherence  to  designing  instruction  focused  upon  larger  content  units.  This  latter  move  is,  of 
course,  consistent  with  Gagne  and  Merrill’s  (1990)  advocacy  of  integrated  goals  as  well  as  Gagne’s  initial  position 
favoring  hierarchies  that  focus  on  larger  intellectual  skills  rather  than  discrete  pieces  of  knowledge.  The  increased  use  of 
analysis  is  a direct  extension  of  Gagne  thinking,  even  though  there  may  be  some  debate  as  to  the  legitimate  focus  of  such 
analytic  activity. 

On  the  other  hand,  the  findings  of  recent  designer  decision-making  research  draw  a picture  of  expert  designers 
working  in  a far  less  structured  manner,  responding  spontaneously  to  situations  which  “trigger  opportunistic  excursions 
that  yield  unexpected  insights  into  the  problem”  (Tripp,  1994,  p.  1 17).  However,  Tripp  also  cites  other  research  that 
shows  designers  using  a combination  of  systematic  analysis  and  opportunistic  tactics.  While  Gagne  has  not  specifically 
addressed  this  topic,  it  seems  likely  that  he  would  support  the  latter  approach. 

Most  designers  today  are  under  great  pressure  to  produce  a product  in  a shorter  period  of  time  than  one  would 
think  realistic.  While  they  know  those  procedures  that  are  “textbook  perfect”,  they  face  daily  demands  that  they  take 
shortcuts.  The  first  steps  to  be  slashed  typically  relate  to  evaluation  and  a detailed  analysis  of  both  content  and  needs. 
However,  many  designers  are  seeking  ways  to  adhere  to  the  time-proven  methods  and  still  be  realistic  in  a business  sense. 
Rather  than  sacrificing  pre-design  analysis  standards,  one  salvation  may  be  the  use  of  computer-based  design  tools. 

Those  tools  that  relate  to  content  analysis,  however,  are  based  for  the  most  part  on  Gagne  techniques.  Early  work  in  this 
area  has  been  completed  by  Gustafson  and  Reeves  (1990)  and  Merrill,  Li,  and  Jones  (1990). 

Another  current  effort  to  increase  the  efficiency,  as  well  as  the  effectiveness,  of  the  typical  design  task  involves 
the  use  of  rapid  prototyping.  Tripp  and  Bichelmeyer  (1990)  describe  this  methodology  as  one  in  which  “after  a succinct 
statement  of  needs  and  objectives,  research  and  development  are  conducted  as  parallel  processes  that  create  prototypes, 
which  are  then  tested  and  which  may  or  may  not  evolve  into  a final  product”  (p.  35).  As  with  the  use  of  computer-based 
design,  rapid  prototyping  builds  upon  traditional  design  practice  although  the  stages  are  not  linear  in  nature  (Jones,  Li,  & 
Merrill,  1992).  Content  analysis  in  the  Gagne  tradition,  however,  is  central  to  the  early  rapid  prototyping  stages  in  a 
similar  fashion  to  its  use  in  conventional  systematic  design.  Other  traditional  design  tools,  many  of  which  were 
introduced  by  Gagne,  also  have  the  ability  to  transcend  current  changes  in  our  work  environments.  Many  of  these 
changes  are  precipitated  by  the  availability  of  the  new  technologies,  and  even  though  Gagne’s  orientaion  is  not 
dominanted  by  technology  it  nonetheless  accommodates  technology’s  capabilities. 

The  Continuing  Dominance  of  the  Events  of  Instruction 

Another  important  tool  for  designers  has  been  the  use  of  Gagne’s  Events  of  Instruction  (Gagne,  1985,  1988; 
Gagne,  Briggs,  & Wager,  1992).  The  “Events”  serve  as  a conceptual  model  for  the  design  of  lessons,  the  selection  of 
instructional  strategies,  and  the  sequencing  of  instruction.  In  essence  the  “Events”  summarize  much  of  the  key  research 
related  to  instruction,  including  factors  such  as  motivation,  perception,  feedback,  reinforcement,  individual  differences, 
retention,  and  transfer.  They  provide  a framework  for  creating  those  external  conditions  that  promote  learning. 

Inherent  in  the  “Events”  is  the  notion  of  designer  control  of  instructional  options.  While  this  is  at  odds  with 
some  constructivist  theory,  it  is  nonetheless  consistent  with  the  vast  majority  of  the  design  practice  in  education  and 
training  environments  today.  The  “Events”  have  been  used  regardless  of  the  delivery  medium,  encompassing  everything 
from  stand-up  training  to  computer-based  instruction.  For  many  expert  designers  the  “Events”  are  now  an  internalized 
model  that  guides  their  work  on  a seemingly  intuitive  level.  This  is  reflective  of  Duffy  and  Jonassen’s  (1991 ) assertion 
that  “while  instructional  designers  typically  may  not  have  the  time  or  support  to'explicitly  apply  a theory  of  learning 
during  a design  or  development  task,  the  theory  is  nonetheless  an  integral  part  of  the  instruction  that  is  produced”  (p.  7). 

While  some  may  disagree  on  the  particular  strategy  that  is  best  for  a given  situation,  there  is  little  disagreement 
with  the  Events  of  Instruction  themselves  because  they  summarize  key  stages  in  the  instructional  process  which  have 
been  repeatedly  validated  in  the  research  literature.  For  example,  conscious  lear  ning  in  a formal  instructional 
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environment  requires  attending  to  the  topic  and  guidance,  as  well  as  reinforcement.  While  there  are  various  strategies  for 
accomplishing  these  tasks,  one  must  be  selected.  As  such,  the  general  framework  provided  by  the  Events  remains 

constant 

To  a great  extent  the  “Events”  framework  is  likely  to  remain  useful  even  in  situations  which  have  student- 
controlled  sequencing  of  learning  activities,  as  is  more  frequently  the  case  in  computer  based  instruction  This  is  because 
the  instructional  events  still  need  to  be  programmed  and  available  for  learner  use.  While  it  is  likely  that  multip  e 
strategies  and  activities  will  be  incorporated  into  a particular  piece  of  instruction,  each  of  the  various  functions  of 
instruction  (as  suggested  by  the  “Events”  model)  must  still  be  accommodated.  Such  structure  is  as  appropriate  for 
designing  individualized  environments,  as  it  has  proven  to  be  for  the  design  of  teacher-directed  instruction. 

The  Stability  of  Gagne's  Orientation  to  Practice 

Gagne  has  consistently  argued  that  instructional  design  practice  should  be  based  upon  what  we  know  about 
human  learning.  This  position  is  seemingly  axiomatic..  Since  it  is  unclear  to  most  that  the  field  currently  has  a 
completely  accurate  view  of  all  human  learning,  it  is  possible  that  the  stability  of  Gagne’s  orientation  to  design  practice 
is  assured  because  of  our  tendency  to  combine  ideas  from  a variety  of  plausible  explanations  of  the  learning  process. 

However  practice  techniques,  even  when  based  upon  complex  theory,  often  tend  to  be  streamlined  and 
simplified.  Perhaps  this  accounts  for  the  fact  that  there  has  been  less  debate  related  to  design  practice  than  theory,  and  for 
many  of  Gagne’ s practice  techniques  to  remain  current  even  in  the  midst  of  great  theoretical  debate.  Bednar,  et  al.  ( 1 ) 

are  not  satisfied  with  the  field’s  tendency  to  create  a patchwork  collection  of  tools  and  techniques  that  have  been  abstracted 
from  different  (and  often  conflicting)  theories  being  used  in  a given  design  project.  They  argue  that  this  eclectic  approach 
does  not  produce  the  most  effective  instruction. 

In  any  case,  Gagne’s  basic  orientation  has  become  ensconced  in  design  tradition,  even  with  the  emergence  of 
new  theory  Most  trained  instructional  designers  select  their  design  focus  depending  upon  the  nature  of  the  learning  task, 
and  are  likely  to  continue  this  practice.  Most  will  continue  to  conduct  some  sort  of  pre-design  content  analysis  as  a 
precursor  to  sequencing  and  identification  of  prerequisite  skills  - even  if  they  not  overtly  use  the  learning  hierarchy 
tool  Most  will  continue  to  select  instructional  strategies  based  upon  a general  Events  of  Instruction  framework  - even 
if  it  is  internalized  and  not  consciously  used.  To  some  extent  this  begs  the  question  of  whether  expert  designers  who 
demonstrate  alternative  design  decision-making  patterns  are  really  deviating  from  the  Gagne  tradition  or  are  still  using  the 

same  principles.  . ....  . 

If  major  deviations  from  the  Gagne  orientation  do  occur,  it  is  typically  because  time  limitations  are  posing 

barriers  to  their  use.  Such  pressures  are  leading  to  a reexamination  of  design  practice.  However,  most  are  directing  their 
efforts  towards  ways  of  increasing  design  efficiency  using  the  same  basic  orientation,  rather  than  making  a sharp  reak 
with  past  tradition.  Dick  (1993)  calls  this  process  the  enhancement  of  the  instructional  systems  design  process.  Even 
time  saving  design  models  such  as  Tessmer  and  Wedman’s  (1990)  Layers  of  Necessity  approach,  which  suggests  a way  to 
streamline  the  process  given  the  demands  of  a given  situation,  does  not  radically  change  the  fundamental  orientation  to 
design.  Thus,  it  seems  that  the  basic  Gagne  approach  will  continue  to  provide  direction  to  the  field,  even  given  the 
likely  changes  and  advancements  in  design  tools  and  techniques. 

Conclusion 

Robert  Gagne  has  substantially  shaped  a new  field  of  instructional  design  during  his  career  as  a psychologist. 

He  made  enormous  contributions  and  had  an  enormous  impact  as  both  a researcher  and  a practitioner.  While  design  has 
been  called  a “linking  science”,  Gagne  himself  has  also  served  a linking  role  throughout  his  career.  He  has  linked  the 
heyday  of  behavioral  psychology  with  the  dominance  of  cognitive  psychology.  He  has  linked  the  field’s  emphasis  on 
designing  educational  programs  for  children  with  an  emphasis  on  designing  training  programs  for  adults  in  the  military 
and  in  business  settings.  He  has  linked  basic  learning  research  to  applied  educational  research.  He  has  linked  theory  to 

practice.  ^ ^ ^ ^ y£ars  of  active  work>  Gagne  explored  the  complex  processes  of  learning  and  instruction,  and 

explained  them  to  generations  of  designers  in  a simple,  understandable  way.  In  the  process  he  has  demonstrated  his  true 
genius.  Gagne’s  work  was  spurred  on  by  important  social  events  that  highlighted  its  importance  and  need  for  the  general 
public  rather  than  only  a small  intellectual  community.  There  was  an  urgent  need  for  efficient,  effective  training  early  in 
World  War  II.  The  Sputnik  crisis  in  the  1950’s  highlighted  the  need  for  American  schools  to  reinforce  mathematics  and 
science  education.  American  corporations  looked  to  education  and  training  as  an  avenue  to  retool  their  workforces  and 
meet  foreign  competition.  His  research  and  the  successful  application  of  this  research  in  a variety  of  settings  provided 
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evidence  of  its  relevance  and  practicality.  In  addition,  its  legitimacy  was  rooted  in  scientific  authority  and  superior 
academic  credentials. 

Gagne’s  ideas  were  part  of  other  prominent  intellectual  movements  over  the  years,  including  both  behavioral  and 
cognitive  psychology,  general  systems  theory,  and  the  early  explorations  into  the  nature  of  instructional  theory.  He  was 
a contemporary  of  other  giants  of  the  world  of  education  scholarship,  including  persons  such  as  Benjamin  Bloom,  Jerome 
Bruner,  John  Carroll,  Robert  Glaser,  and  Ralph  Tyler.  In  retrospect,  there  was  a social  and  intellectual  climate  in  the 
United  States  that  was  conducive  to  the  proliferation  and  acceptance  of  Gagne's  work. 

Today,  the  field  of  instructional  design  has  grown.  It  has  many  areas  of  specialization,  many  delivery  options, 
and  many  alternative  theoretical  perspectives  that  command  considerable  support.  Furthermore,  there  are  far  more  people 
involved  in  the  field.  This  growth  in  itself  is  testament  to  Gagne’s  work.  However,  this  more  complex  environment 
may  greatly  reduce  the  possibilities  of  one  person  alone  exercising  the  same  over-arching  dominance  of  a Robert  Gagne. 

Yet  Gagne’s  influence  is  surely  attributed  to  more  than  “being  in  the  right  place  at  the  right  time”.  Ultimately, 
his  influence  is  a product  of  the  power  of  his  ideas.  His  influence  is  a product  of  those  seemingly  simple  principles 
which  most  of  us  are  still  re-examining  and  continuing  to  find  that  they  provide  new  meaning  and  new  direction.  Is  that 
not  the  power  of  an  intellectual  legacy?  The  ideas  continue,  and  new  generations  meet  them,  become  engaged,  and  have 
yet  another  “Eureka”  experience. 
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Abstract:  The  purpose  of  this  study  was  to  explore  how  adult  users  interact  and  learn  during  a computer- 
based  simulation  given  visual  and  verbal  forms  of  feedback  coupled  with  embedded  elaborations  of  the 
content.  A total  of  52  college  students  interacted  with  a computer-based  simulation  of  Newtons  laws  of 
motion  in  which  they  had  control  over  the  motion  of  a simple  screen  object  — a ball.  Two  simulation 
conditions  were  studied,  each  differing  in  how  the  feedback  of  the  ball's  speed,  direction,  and  position  was 
represented:  visual  feedback  consisted  of  animated  graphics  and  verbal  feedback  consisted  of  numeric 
displays.  In  addition,  half  of  the  simulations  were  supplemented  with  elaborations  of  the  content  modeled 
by  the  simulation  in  order  to  investigate  how  to  promote  referential  processing,  a key  component  of  dual 
coding  theory.  Results  showed  significant  differences  for  both  the  use  of  the  elaborations  and  simulations 
containing  visual  feedback  in  helping  subjects  gain  both  tacit  and  explicit  understanding  of  the  science 
principles.  In  addition  to  the  quantitative  analysis,  a qualitative  analysis  was  also  conducted  with  12 
additional  subjects.  This  analysis  revealed  interesting  trends  in  how  some  subjects  valued  and  used  the 
elaborations  and  the  two  feedback  representations  of  the  simulation. 

Physics  has  long  been  regarded  as  one  of  the  most  abstract  and  difficult  subjects  to  learn.  A common  belief  is 
that  the  students  who  do  well  in  physics  have  a special  aptitude  for  learning  science  and  mathematics.  It  is  easy  to 
understand  how  many  might  believe  that  the  only  students  who  are  going  to  "need"  physics  are  future  engineers  and 
physicists.  However,  instead  of  reserving  physics  for  a gifted  few,  many  feel  that  technology  offers  the  chance  to  change 
commonly  help  perceptions  about  who  can  or  should  learn  physics  (White,  1993).  For  example,  complex  systems  such 
as  physics  can  easily  be  modeled  on  desktop  computers.  Computer  simulations  make  complex  systems  accessible  for 
students  of  varying  ages,  abilities,  and  learning  levels.  The  computer,  instead  of  the  student,  can  assume  responsibility 
of  processing  the  underlying  mathematics  in  order  to  let  the  student  begin  exploring  a complex  system  by  first  focusing 
on  conceptual  understanding. 

Of  course,  many  challenges  to  the  effective  design  of  simulations  remain.  Among  the  most  difficult  are 
questions  about  the  design  of  a simulation's  interface  (Schneiderman,  1987).  For  example,  designers  currently  can 
include  a wide  range  of  visual,  textual,  and  aural  elements  in  the  development  of  simulations.  As  the  range  of  available 
design  options  increases,  so  too  does  the  complexity  of  design  decisions.  One  of  the  most  important  considerations  in  a 
simulation's  interface  design  is  how  to  provide  meaningful  feedback  to  the  user.  Feedback  has  long  been  regarded  by 
cognitive  psychologists  as  one  of  the  most  critical  sources  of  information  to  assist  learners  in  restructuring  their 
knowledge  and  supporting  their  metacognitive  processes  (Kulhavy  & Wager,  1993).  Given  the  range  of  ways  computers 
can  represent  feedback  in  a simulation,  research  is  needed  to  ensure  that  design  decisions  are  made  based  on  the 
psychological  needs  of  the  individual  user  and  not  simply  on  what  the  computer  is  capable  of  doing. 

Much  research  demonstrates  that  the  way  information  is  represented  matters  greatly  in  the  learning  process,  at 
least  for  memory  tasks  (Clark  & Paivio,  1991;  Paivio,  1990;  Paivio,  1991).  Research  indicates  that  pictures  are  superior 
to  words  for  remembering  concrete  concepts  (Sadoski,  Goetz  & Fritz,  1993).  Among  the  various  theories  that  have  been 
proposed  to  explain  this,  Paivio's  dual  coding  theory  appears  to  have  the  strongest  empirical  support  (Anderson,  1978; 
Sadoski  & Paivio,  1994;  Sadoski,  Paivio  & Goetz,  1991).  Dual  coding  theory  divides  cognition  into  two  processing 
systems  — one  visual  and  one  verbal.  Although  the  research  supporting  dual  coding  theory  is  based  almost  exclusively 
on  evidence  derived  from  supplementing  printed  text  with  visuals  (including  paired  associative  tasks)  (e.g.,  Sadoski, 

Goetz  & Avila,  1995;  Sadoski  & Paivio,  1994),  this  theory  also  holds  promise  in  guiding  research  in  computer-based 
multimedia  environments  (Mayer  & Sims,  1994;  Rieber,  in  press). 

Dual-coding  theory  predicts  three  separate  levels  of  processing  within  and  between  the  visual  and  verbal  systems: 
representational,  associative,  and  referential.  Representational  structures  (either  visual  or  verbal)  are  formed  depending  on 
the  nature  of  incoming  information  (i.e.,  visual  and  verbal  information  from  the  environment  triggers  the  visual  and 
verbal  systems  respectively).  Associative  processing  leads  to  connections  constructed  within  either  the  visual  or  verbal 
systems,  whereas  referential  processing  leads  to  connections  made  between  the  visual  and  verbal  systems.  Referential 
processing  is  particularly  important  because  dual  coding  theory  predicts  that  learning  will  be  enhanced  when  information 
is  encoded  in  both  systems  (i.e.,  dually  coded).  Information  that  is  dually  coded  has  twice  the  chance  to  be  retrieved  and 
used  (Kobayashi,  1986).  Instruction  that  promotes  dual  coding  has  obvious  advantages. 

Our  past  research  has  shown  that  the  way  feedback  is  represented  also  matters  when  learning  from  simulations  of 
physical  science  concepts  and  principles  (i.e.,  laws  of  motion).  Subjects  increased  their  tacit  knowledge  of  physics  when 
they  interacted  with  a physics  simulation  given  visual  feedback,  but  they  were  unable  to  demonstrate  increased  explicit 
understanding  based  on  the  way  the  feedback  was  represented  (see  Rieber,  in  press;  Rieber  et  al.,  in  press).  Tacit 
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understanding  was  measured  by  subjects'  performance  in  a gamelike  activity  whereas  explicit  understanding  was  measured 
using  a traditional  performance  test  (i.e.,  multiple-choice  question  format).  The  increase  in  tacit  learning  given  visual 
feedback  indicated  that  representational  and  associative  processing  occurred  almost  exclusively  within  the  visual  system. 
Subjects'  difficulty  in  acquiring  explicit  understanding  of  the  physics  principles  modeled  by  the  computer  was  attributed 
to  the  highly  interactive  nature  of  the  discovery-based  simulation.  Simulations  that  model  physical  phenomena  (such  as 
physical  science)  may  not  provide  the  learner  with  sufficient  time  or  guidance  for  interpreting  the  continual  stream  of 
feedback  by  both  the  visual  and  verbal  systems.  In  other  words,  the  "video  game-like"  quality  of  the  simulation  may 
have  interfered  with  referential  processing. 

The  purpose  of  this  study  was  to  investigate  ways  to  facilitate  or  enhance  referential  processing  as  a user 
interacts  with  a computer  simulation.  Our  previous  research  used  a pure  discovery-based  approach  — no  instruction  was 
included  or  embedded  in  the  simulation.  While  the  highly  experiential  nature  of  open-ended  simulations  appear  beneficial 
in  many  ways,  it  does  not  seem  to  adequately  promote  reflection  of  the  science  principles.  Reflection  appears  to  be  an 
important  component  for  referential  processing.  Therefore,  the  use  of  brief  elaborations  of  the  science  principles 
embedded  throughout  the  user's  experience  with  the  simulation  was  studied  in  this  research.  It  was  hypothesized  that 
supplementing  the  simulation  with  elaborations  of  the  content  would  facilitate  all  three  types  of  processing  predicted  by 
dual  coding  theory  for  explicit  learning.  Also,  since  previous  research  suggests  the  apparent  dominance  of  the  visual 
system  during  a user's  interaction  with  simulations  similar  to  these,  it  was  also  hypothesized  that  the  embedded 
elaborations  would  promote  more  referential  processing  when  subjects  were  given  visual  instead  of  verbal  feedback. 

A variety  of  data  sources  were  used  in  this  study.  Traditional  performance  measures  (e.g.,  question-based 
pretests  and  posttests)  were  used  to  assess  subjects'  explicit  understanding  of  the  science  principles  modeled  in  the 
simulation.  However,  such  formal  tests  do  not  assess  other  levels  of  understanding  which  are  embedded  in  a task.  For 
example,  bringing  a car  to  a smooth  controlled  stop  requires  an  extensive  understanding  of  many  motion  principles. 
However,  this  understanding  remains  situated  in  the  act  of  driving  — the  individual  may  not  be  able  to  explicitly  describe 
the  physical  relationships  at  work.  Although  we  recognize  that  a learner's  ability  to  transfer  conceptual  understanding 
from  one  task  to  another  (such  as  to  a posttest)  remains  an  important  indication  of  learning,  this  study  also  used  a 
measure  of  tacit  understanding  found  useful  in  our  earlier  research.  Subjects  were  asked  to  complete  the  simulation  in  a 
game-like  context.  Since  an  understanding  of  the  motion  principles  is  necessary  to  be  successful  at  the  game,  the  game 
score  provides  an  alternative  data  source  useful  when  compared  with  the  subjects’  scores  on  traditional  performance 
measures.  Finally,  this  study  also  included  both  a quantitative  and  qualitative  phase.  Rather  than  arguing  in  favor  of  one 
research  methodology  over  another,  we  believe  that  the  combination  of  methodologies  provides  a more  complete 
understanding  that  either  could  provide  separately. 

Method 

The  quantitative  methods  and  results  of  this  study  will  be  reported  first,  followed  by  the  qualitative  methods  and 

results. 

Subjects 

A total  of  52  subjects  participated  in  the  experimental  phase  of  this  study.  Subjects  were  upperclass 
undergraduate  students  enrolled  in  an  introductory  computer  education  course.  Participation  was  voluntary,  though  extra 
credit  in  the  course  was  provided  to  students  as  incentive  to  participate. 

Materials 

The  materials  consisted  of  a computer-based  simulation  of  Newton's  laws  of  motion.  Subjects  had  direct  control 
over  the  motion  of  a simulated,  free-floating  object  (called  simply  a "ball").  The  simulation  was  presented  in  a game-like 
context  with  the  goal  of  moving  the  ball  to  a specific  screen  location  (called  the  "target").  All  52  subjects  were  given  a 
total  of  30  trials.  It  was  anticipated  that  as  subjects  gained  mastery  of  the  game,  they  would  become  bored  unless  the 
challenge  was  increased.  For  the  first  20  trials,  all  subjects  merely  had  to  guide  the  ball  to  the  target.  However,  in  the 
final  10  trials,  all  subjects  had  to  guide  the  ball  to  the  target  and  bring  the  ball  to  a stop  while  inside  the  target. 

Subjects  were  able  to  move  the  object  in  two  dimensions.  They  controlled  the  motion  of  the  object  by  pressing 
one  of  four  screen  buttons  that  applied  an  impulse  force  to  it,  similar  to  a "kick,"  in  one  of  four  directions:  left,  right, 
up,  or  down.  The  magnitude  of  the  force,  or  kick,  did  not  vary.  No  other  forces  (e.g.,  gravity  and  friction)  were  included 
in  the  simulation.  The  computer  calculated  the  resulting  motion  of  the  object  (i.e.,  position,  speed,  and  direction)  and 
reported  this  information  back  to  the  user  in  real-time.  In  other  words,  the  computer  calculated  the  motion  of  the  object 
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about  as  fast  as  the  user  interacted  with  the  simulation.  The  computer  provided  this  information  to  the  user  in  one  of 
two  ways,  either  as  visual  feedback  or  as  verbal  feedback.  Visual  feedback  contained  a graphic  of  the  target  and  an 
animated  graphic  of  the  ball  while  verbal  feedback  consisted  of  numerical  readouts  of  the  screen  positions  of  the  ball  and 
target.  Half  of  the  subjects  were  given  visual  feedback  and  the  other  half  verbal  feedback.  Examples  of  these  two  forms 
of  feedback  are  illustrated  in  Figures  1 and  2. 


Figure  1.  Snapshot  of  the  computer  screen  during  the  simulation  in  which  visual  feedback  was 
provided.  Feedback  about  the  ball's  position  was  displayed  by  animating  the  ball  on  the  computer 
screen. 
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Figure  2.  Snapshot  of  the  computer  screen  during  the  simulation  in  which  verbal  feedback  was 
provided.  Feedback  about  the  balls  position  was  displayed  only  in  numerical  form. 


Subjects  were  also  randomly  assigned  to  one  of  two  levels  of  Elaboration  (Yes,  No).  Elaboration  consisted  of 
five  separate  explanative  frames  that  explicitly  described  the  motion  principles  using  a combination  of  text  and  animated 
graphics.  The  five  frames  were  presented  in  sequence,  one  elaboration  frame  after  every  two  simulation  trials  (e.g., 
elaboration  frame  1 after  simulation  try  2,  elaboration  frame  2 after  simulation  try  4,  etc.).  Therefore,  each  of  the  five 
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elaboration  frames  was  presented  three  times  throughout  the  simulation  trials.  Half  of  the  subjects  were  given  these 
elaborations  and  half  were  not.  An  example  of  one  of  the  elaborations  is  illustrated  in  Figure  3. 


Helpful  Hint  #2 

(You  wil  see  this  and  other  hints  after  every  other  akrulation) 

This  simulation  is  based  on  Newton's  laws  of  motion. 


Newton's  second  law  says  that  the  speed  of  an  object 
depends  on  size  of  the  force  acting  upon  it. 


Therefore,  an  object  kicked  two  times  to  the  right  would 
move  at  a speed  twice  as  fast  as  a ball  kicked  only  once 
(again  assuming  there  were  no  other  forces  present  like 
gravity  or  friction). 


(.’licktiin wlim'  lo  fouiimie  ' 


Figure  3.  An  example  of  one  of  the  5 embedded  elaborations.  The  two  balls  were  animated 
concurrently. 

Dependent  Measures 

Performance.  A 20~item  test  was  used  to  measure  subjects'  explicit  understanding  of  Newtonian  motion 
principles.  The  questions  were  designed  to  test  for  rule-using  learning  outcomes  (i.e.,  principles)  (as  defined  by  Gagnd, 
Briggs  & Wager,  1992).  Multiple-choice  questions  (1  answer  and  4 distractors)  were  used  as  the  testing  format. 
Representative  questions  are  shown  in  Figure  4.  KR-20  reliability  was  .85. 

Game  Score.  The  time,  in  seconds,  taken  by  subjects  to  successfully  complete  the  game  was  used  as  a scoring 
feature.  A subject's  score  for  any  one  simulation  trial  was  equal  to  the  number  of  seconds  elapsed  at  the  moment  the 
game  was  completed.  Each  trial  had  a time  limit  of  two  minutes.  If  time  ran  out  before  the  subjects  successfully 
completed  the  game,  the  computer  automatically  signaled  the  end  of  the  simulation  and  a score  of  120  was  recorded  for 
that  try. 

Interactivity.  The  total  number  of  times  subjects  clicked  either  the  “right,”  “left,”  “up,”  or  “down”  button 
during  each  simulation  was  recorded  by  the  computer. 

Frustration.  After  each  simulation  trial,  subjects  were  asked  to  rate  their  level  of  frustration  on  a scale  of  0 to  8 
where  0 was  “no  frustration,”  8 was  “extreme  frustration.” 
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Question  4 of  20 

Pretend  that  there  is  no  gravity  or  friction. 

A ball  is  kicked  three  times  to  the  right.  Which  of  the 
following  will  make  the  ball  go  in  the  opposite 

1 . 1 kick  to  the  left 

2.  2 kicks  to  the  left 

3.  3 kicks  to  the  left 

4.  4 kicks  to  the  left 

5.  None  of  these  will  make  it  move  in  the  opposite 
direction. 

t 


Press  or  cltck  on  the  number  of  the  answer  Of  your  choice. 


Question  6 of  20 


Pretend  that  there  is  no  gravity  or 
friction. 

The  ball  is  kicked  once  to  the  top  as 
shown  in  the  diagram. 

Choose  the  best  place  to  give  the  ball 
another  kick  to  the  right  in  order  to  hit 
the  square  target  (assuming  the  force  is 
the  same  for  each  kick). 


1.^ 

2.4*» 


5.  None  of  these  will 
make  the  ball  hit  the 
target. 


Press  or  click  dn  the  number  of  the  answer  of  your  choice 


Figure  4.  Two  representative  questions  from  the  pretest/posttest.  (The  correct  answer  to  the  top 
question  is  "4"  and  the  correct  answer  to  the  bottom  question  is  ”3".) 

Procedures 

All  of  the  simulation  conditions  and  testing  were  administered  by  the  computer.  Subjects  were  randomly 
assigned  to  one  of  the  four  conditions  (i.e.,  visual  or  verbal  feedback  with  or  without  embedded  elaborations)  as  they 
reported  to  the  computer  lab.  The  computer  immediately  administered  the  20-item  pretest.  Subjects  were  then  given  two 
practice  trials  with  the  simulation  as  an  orientation  to  the  task.  Subjects  then  were  given  a total  of  30  attempts  with 
their  respective  simulation  condition  with  or  without  embedded  elaborations.  After  each  simulation  try,  the  computer 
surveyed  subjects  on  their  level  of  frustration.  Immediately  upon  completion  of  the  simulation  activities,  the  computer 
automatically  administered  the  posttest  consisting  of  the  same  20  multiple-choice  items.  Approximately  one  hour  was 
needed  to  complete  the  experiment. 

Design 

This  study  used  a 2x2  factorial  design  involving  two  levels  of  two  between-subjects  factors:  Feedback  (visual, 
verbal)  and  Elaboration  (yes,  no).  Statistical  procedures  included  separate  Analysis  of  Variance  (ANOVA)  tests  on  each 
of  the  dependent  measures  (a  repeated  ANOVA  was  used  on  the  pretest/posttest  performance  measure). 
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Results 


Performance.  Percentage  means  and  standard  deviations  are  contained  in  Table  1.  A significant  interaction  was 
found  between  Elaboration  and  Performance,  F(l,48)=9.55,  pc.Ol,  MSerror=190.51 . All  subjects  scored  similarly  on  the 
pretest  (mean=53.2%),  however,  subjects  provided  with  embedded  elaborations  scored  higher  on  the  posttest 
(mean=83.8%)  than  subjects  who  were  not  given  the  elaborations  (mean=70.7%).  There  was  also  a significant 
interaction  between  Feedback  and  Performance,  F(l,48)=5.0,/x.05,  MSerrOr=190.51.  Again,  all  subjects  scored 
similarly  on  the  pretest,  however,  those  provided  with  visual  feedback  scored  higher  on  the  posttest  (mean=84.8%)  than 
those  provided  with  verbal  feedback  (mean=69.8%).  No  interaction  was  found  between  Feedback  and  Elaboration:  the 
embedded  elaborations  were  equally  beneficial  to  subjects  regardless  of  which  feedback  type  (visual  or  verbal)  they  were 
given  in  the  simulation. 

Given  the  main  effects  for  both  Elaboration  and  Feedback,  a separate  post  hoc  analysis  was  conducted  on  the 
simple  effects  between  the  four  cell  means  using  only  the  posttest  scores  to  test  for  the  additive  effects  of  the  elaborations 
combined  with  visual  feedback.  Results  showed  a significant  difference  between  the  four  cell  means,  F(3,48)=4.4,  pc.Ol, 
MSerror^O^.  Follow-up  tests  showed  that  subjects  given  both  visual  feedback  during  the  simulation  and  the 
embedded  elaborations  scored  significantly  higher  (mean=93.5%)  than  all  other  groups  (mean=7 1 .9%),  as  illustrated  in 
Figure  5.  Subjects  given  visual  feedback  combined  with  embedded  elaborations  attained  near  mastery  on  the  posttest. 

Game  score.  A significant  main  effect  was  found  for  Feedback,  F(l,48)=46.2,  pc.OOOl,  MSerror=223974.9. 
Subjects  given  visual  feedback  scored  better  on  the  gaming  activity  than  subjects  provided  with  verbal  feedback.  This 
result  is  also  consistent  with  previous  research.  Subjects  level  of  tacit  learning  was  facilitated  more  with  visual  feedback 
than  verbal  feedback.  However,  no  significant  differences  were  found  for  Elaboration:  explicit  information  about 
Newton's  laws  of  motion  did  not  affect  subjects'  game  scores  in  any  way. 

Interactivity.  No  main  effects  or  interactions  were  found  for  Elaboration  or  Feedback.  Subjects'  frequency  of 
interactivity  (i.e.,  mouse  clicks)  was  similar  for  all  subjects  in  all  treatment  groups. 

Frustration.  A significant  main  effect  was  found  for  Feedback,  F(  1,48)=  12.9,  /K.001,  MSerror=2501.2. 
Subjects  provided  with  visual  feedback  were  significantly  less  frustrated  than  subjects  provided  with  verbal  feedback. 
However,  there  was  no  main  effect  for  Elaboration:  the  presence  or  absence  of  the  elaboration  did  not  affect  subjects' 
frustration  levels. 

Qualitative  Methods 

Triangulation  is  regarded  as  one  important  way  to  strengthen  the  understanding  of  phenomena  under  study.  This 
can  be  accomplished  by  using  different  kinds  of  methods  or  data,  such  as  a combination  of  both  quantitative  and 
qualitative  approaches  (Mathison,  1988;  Patton,  1990).  The  qualitative  method  used  in  this  study  was  designed  to 
complement  the  quantitative  analysis  in  order  to  enhance  the  validity  of  our  findings. 

Interview,  coupled  with  participant  observation,  were  the  two  primary  methods  used  to  collect  the  qualitative 
data.  An  unstructured  interview  protocol  was  determined  in  advance  but  each  interviewer  was  free  to  take  the  interview  in 
any  direction  deemed  necessary.  Each  member  of  the  research  team  acted  as  an  interviewer  for  at  least  one  subject. 
Subjects  were  also  asked  to  use  a "think  aloud"  protocol  to  verbalize  freely  what  they  were  thinking  throughout  their 
experience.  The  goal  was  to  "get  inside  the  participant's  head"  and  elicit  what  they  were  doing  and  why. 
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Table  1 

Mean  Percentage  Scores  and  Standard  Deviations  for  Performance _ 


Performance 

Feedback 

Pretest 

Posttest 

Visual 

M 

With  Embedded  Elaborations 
55.0 

93.5 

SD 

18.6 

9.0 

n 

13 

13 

Verbal 

M 

47.7 

74.2 

SD 

27.1 

28.0 

a 

13 

13 

Visual 

M 

Without  Embedded  Elaborations 
54.2 

76.2 

SD 

18.8 

12.4 

n 

13 

13 

Verbal 

M 

55.8 

65.4 

SD 

17.9 

24.9 

a 

13 

13 

Twelve  subjects  participated  in  the  qualitative  research  (these  subjects  were  drawn  from  the  same  population,  but 
they  did  not  participate  in  the  quantitative  phase).  These  subjects  interacted  with  a modified  version  of  the  simulation 
used  in  the  quantitative  study.  Subjects  had  complete  control  over  the  selection  (when  and  which)  of  the  embedded 
elaborations.  Participants  were  also  given  total  control  over  which  feedback  type  was  chosen  for  the  first  several  trials. 
Thereafter,  the  interviewer  negotiated  with  the  subject  as  to  which  feedback  type  would  be  chosen  in  order  to  avoid  having 
a subject  experience  only  one  feedback  type  (such  as  visual).  In  order  for  participants'  experience  to  closely  match  that  of 
subjects  in  the  quantitative  study,  participants  were  expected  to  complete  at  least  30  trials.  Participants  in  the  qualitative 
study  were  also  required  to  complete  the  pretest  and  posttest.  No  help  was  given  to  participants  on  the  pretest  and 
posttest.  Nevertheless,  during  the  posttest  participants  were  encouraged  to  explain  reasoning  or  confusing  they  were 
experiencing  as  they  answered  the  questions. 

Observational  field  notes  were  taken  by  each  interviewer  while  the  subject  interacted  with  the  simulation.  The 
collected  data  were  then  examined  by  each  interviewer  for  concepts  and  categories  that  emerged  from  sessions.  Finally, 
the  research  team  met  as  a group  to  compare,  contrast,  and  synthesize  the  findings. 
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Figure  5.  Means  of  subjects'  posttest  scores  split  by  Elaboration  and  Feedback. 


Results 

Two  themes  were  derived  from  the  qualitative  analysis  related  to  the  embedded  elaborations  and  the  simulation 

itself. 

Embedded  elaborations.  There  was  a wide  variation  in  how  participants  used  and  valued  the  elaborations. 

Almost  all  subjects  thought  that  the  embedded  elaborations  were  beneficial,  but  most  subjects  did  not  feel  that  they  were 
actually  relying  on  the  hints  to  any  great  extent  in  the  simulation.  However,  an  interesting  pattern  was  noticed  in  several 
of  the  participants.  They  seem  to  be  reassured  that  the  elaborations  were  available  even  though  they  did  not  feel  the  need 
to  consult  them  often.  Still  others  were  a little  confused  about  how  the  hints  were  supposed  to  help  them  in  the 
simulation  as  they  did  not  see  a strong  connection.  Most  participants  did  not  initially  choose  to  click  on  the  hints  and 
instead  opted  to  immediately  begin  interacting  with  simulation.  However,  several  participants  turned  to  the  hints  for 
help  if  they  became  disoriented  in  the  gaming  activity.  Finally,  participants  tended  to  consult  the  hints  sequentially  and 
seldom  consulted  the  same  hint  twice. 

Interaction  within  the  simulation.  Almost  all  participants  showed  a strong  preference  for  the  visual  feedback 
early  on.  However,  this  preference  varied  among  individuals  as  they  gained  experience  with  the  simulation.  Two  profiles 
of  participants’  reaction  to  the  feedback  were  observed  which  we’ve  chosen  to  name  "clickers”  and  "strategists.”  Clickers 
did  not  employ  any  apparent  strategies  and  either  seemed  just  to  be  trying  to  complete  the  exercise  or  were  interacting 
with  the  simulation  in  a very  experiential  way  with  little  or  no  reflection.  Strategists,  in  contrast,  clearly  were  involved 
in  building  and  testing  strategies  for  improving  their  ability  to  play  the  game.  Two  distinct  strategies  emerged:  one- 
dimensional versus  diagonal.  The  one-dimensional  strategy  involved  moving  the  ball  along  one  dimension  and  stopping 
it  when  it  lined  up  with  the  target.  They  then  proceeded  to  move  the  ball  to  the  target  by  way  of  the  other  dimension. 
The  one-dimensional  strategy  seemed  to  work  equally  well  in  either  feedback  situation.  In  contrast,  the  diagonal  strategy 
involved  getting  the  ball  to  move  along  one  dimension  and  then  carefully  timing  when  to  give  the  ball  another  kick 
along  the  other  dimension.  Since  the  ball  had  not  come  to  a stop,  the  ball  resulted  in  a diagonal  motion.  This  strategy 
tended  to  work  well  given  visual  feedback,  but  was  often  disorienting  when  given  verbal  feedback,  especially  if  subjects 
missed  the  target  on  the  first  try.  As  to  the  relative  benefits  of  these  two  types  of  feedback,  one  participant  commented 
that  "you  have  to  think  with  textual  feedback,"  while  in  graphical  feedback  ’’you  can  concentrate  more  on  concepts 
without  worrying  about  techniques."  This  comment  reflects  well  the  experiential  nature  of  the  visual  feedback.  It  also 
demonstrates  the  tendency  by  many  strategists  to  eventually  switch  their  preference  to  the  verbal  feedback  because  it  was 
more  consistent  with  strategies  they  were  developing  plus  it  provided  an  extra  challenge  (most  were  getting  bored  after 
about  1 5 tries). 


Discussion 

The  purpose  of  this  study  was  to  investigate  ways  to  facilitate  or  enhance  an  individual’s  learning  of  physics 
principles  while  interacting  with  a computer  simulation  using  a discovery-based  approach.  Many  facets  of  learning  were 
studied  in  this  research,  such  as  tacit  versus  explicit  understanding  as  well  as  patterns  of  interactivity  and  frustration. 
While  computers  afford  the  design  of  highly  interactive  open-ended  learning  environments  such  as  simulations,  decisions 


about  how  to  design  the  interface  of  a simulation  are  often  made  with  little  understanding  of  ^ow  the  user  will  perceive, 
process,  and  interpret  the  resulting  feedback  that  the  simulation  provides.  This  study  was  designed  to  explore  the  effects 
of  some  of  the  most  basic  interactive  attributes  of  simulations  on  cognition. 

Previous  research  on  discovery  learning  in  a computer  simulation  has  shown  that  although  the  way  a 
simulation’s  feedback  is  represented  can  greatly  influence  subjects’  performance  in  game-like  activities,  differences  in 
feedback  did  little  to  promote  explicit  understanding  of  the  science  principles  necessary  on  traditional  performance 
measures  such  as  question  and  answer  tests  (Rieber,  in  press).  The  present  study  was  designed  to  follow-up  the  results  of 
Rieber  (in  press),  but  with  some  important  changes.  For  example,  subjects  found  the  content  of  the  simulation  used  in 
Rieber  (in  press)  — acceleration  and  velocity  — very  demanding.  The  difficulty  of  the  task  may  have  interfered  with 
understanding.  In  contrast,  the  simulation  used  in  the  present  study  involved  a simpler  model  of  Newtonian  mechanics 
and  was  designed  with  greater  sensitivity  to  motivation  — challenge  was  increased  at  about  the  time  subjects  were 
expected  to  begin  getting  bored.  This  study  also  increased  subjects’  experience  with  the  simulation  — subjects  had  30 
separate  trials  with  the  simulation  (as  compared  to  20  in  previous  research).  The  present  study  also  included  one 
additional  factor  — embedded  elaborations  — to  investigate  means  of  enhancing  referential  processing  without  seriously 
interrupting  the  interactive  nature  of  the  simulation  (as  would  a complete  tutorial). 

This  research  was  guided  by  dual-coding  theory  which  predicts  that  more  and  effective  learning  should  result 
when  information  is  encoded  both  visually  and  verbally  and  if  connections  are  made  between  these  visual  and  verbal  codes 
(Paivio,  1990).  Information  that  is  dually  coded  doubles  the  chance  for  retrieval  since  learners  have  two  ways  to  access 
the  information.  According  to  dual  coding  theory  these  connections  are  established  through  referential  processing. 
However,  there  has  been  little  application  of  dual  coding  theory  to  simulations  and  other  interactive  learning 
environments.  The  highly  interactive  nature  of  many  computer  simulations  creates  an  interesting  dilemma.  On  one 
hand,  the  experiential  nature  of  an  educational  simulation  is  very  compelling  — users  often  become  very  active  and 
engaged  in  a simulation,  similar  to  the  experience  of  playing  a video  game.  However,  the  intense  and  demanding 
interactivity  of  many  simulations  may  not  provide  adequate  time  for  the  user  to  carefully  reflect  on  the  principles  being 
modeled  by  the  simulation.  Without  sufficient  guidance  or  time  and  opportunity  for  reflection,  referential  processing 
may  not  take  place.  Therefore,  while  the  simulation  may  lead  to  successful  tacit  learning  (i.e.,  success  at  completing  the 
simulation  activities),  the  simulation  may  actually  hinder  or  interfere  with  explicit  learning.  This  research  tried  to  gain 
insight  to  this  issue  by  studying  the  influence  of  the  way  a simulation's  feedback  is  represented  in  combination  with  the 
embedded  elaborations. 

In  contrast  to  Rieber  (in  press),  visual  feedback  (consisting  of  continually  updated  animated  graphics)  during  a 
simulation  was  more  effective  than  verbal  feedback  in  helping  subjects  gain  both  tacit  and  explicit  learning  of  Newton’s 
laws  of  motion.  Not  only  did  the  subjects  who  were  given  visual  feedback  outperform  other  subjects  on  the  simulation’s 
gaming  activity  (i.e.,  tacit  knowledge),  their  gains  from  the  pretest  to  the  posttest  were  also  greater. 

The  embedded  elaborations  were  very  successful  learning  aids  even  though  they  were  exceedingly  brief  (one 
frame  each).  Subjects  who  given  the  elaborations  gained  significantly  more  explicit  understanding  of  the  science 
principles  than  those  who  were  not  given  the  elaborations.  Not  surprisingly,  the  elaborations  were  of  little  help  to 
subjects  as  they  played  the  game.  From  a dual  coding  perspective,  the  elaborations  in  tandem  with  the  simulations 
promoted  representational,  associative,  and  referential  processing.  The  elaborations  were  equally  beneficial  to  all  subjects 
— there  were  no  differentiated  effects  based  on  the  type  of  feedback  provided.  This  was  surprising.  It  was  expected  that 
subjects  given  the  visual  feedback  would  be  in  a better  position  to  make  meaning  out  of  the  embedded  elaboration  than 
subjects  given  the  verbal  feedback.  Of  course,  the  embedded  elaborations  used  here  contained  both  visual  and  verbal 
elements.  The  elaborations  may  have  influenced  mental  imaging  for  subjects  given  verbal  feedback.  The  qualitative 
results,  however,  suggest  that  although  subjects  thought  the  elaborations  were  generally  beneficial,  they  tended  not  to  use 
the  elaborations  much  because  they  did  not  see  them  as  an  aid  in  achieving  greater  success  at  the  game.  Subjects  largely 
failed  to  focus  on  the  more  global  goal  of  using  the  simulation  and  the  elaborations  as  a means  of  learning  about  the 
laws  of  motion.  The  phenomena  of  users  not  choosing  to  select  options  from  which  they  clearly  could  benefit  is 
reminiscent  of  the  research  on  learner  control  (Clark,  1982;  Milheim  & Martin,  1991;  Steinberg,  1989). 

The  additive  effects  of  the  visual  feedback  coupled  with  embedded  elaborations  deserves  special  attention.  As 
indicated  in  the  post  hoc  analysis  of  the  simple  effects  on  the  posttest,  subjects  given  both  visual  feedback  and 
elaborations  reached  near  mastery  on  the  posttest  (average  score  of  93.5%),  outperforming  all  other  subjects.  These 
results,  though  still  preliminary,  suggest  an  interesting  approach  to  optimizing  the  experience  of  learning  by  simulation. 
Unlike  traditional  approaches  where  simulations  are  usually  used  as  follow-up  practice  activities  to  tutorials  (see,  for 
example,  Alessi  & Trollip,  1991),  it  may  be  possible  to  center  learning  around  the  highly  interactive  and  experiential 
nature  of  a simulation.  Certainly,  there  seem  to  be  cognitive  advantages  to  students  making  meaning  through  personal 
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discovery  and  exploration.  Every  "secret"  they  discover,  recognize,  and  articulate  becomes  their  knowledge,  not  someone 
else’s.  Of  course,  the  promise  of  personally  constructed  knowledge  comes  with  increased  responsibility  on  the  part  of 
each  learner.  Computer  simulations  afford  learners  with  the  chance  to  interact  with  information-laden  representations  of 
complex  domains  (e.g.,  physics,  mathematics,  chemistry,  history,  etc.).  In  many  simulations,  students  face  the  burden 
of  making  meaning  from  a continual  stream  of  information  about  the  physical  properties  of  screen  objects.  These  results 
suggest  a way  to  supplement  a simulation  to  help  students  meet  the  difficult  task  of  learning  in  a simulation. 

It  is  important  to  note  the  task  specific  nature  of  the  visual  representation  of  feedback  produced  by  a computer 
simulation.  A conclusion  of  this  study  is  that  different  representations  will  lead  to  different  outcomes  for  certain  tasks 
and  not  that  visual  feedback  is  generally  "better"  than  verbal  feedback  for  all  tasks.  The  design  challenge  is  to  match  the 
demands  of  the  task  with  appropriate  representations.  Certainly  in  this  study,  the  visual  feedback  was  very  consistent 
with  learning  about  the  laws  of  motion.  However,  as  the  qualitative  data  suggest,  many  "strategists"  often  came  to  prefer 
the  verbal  feedback  after  sufficient  experience  with  the  simulation  because  they  began  to  develop  creative  strategies  to 
further  their  success  and  enjoyment  in  playing  the  game. 

The  lack  of  main  effects  on  interactivity  is  in  contrast  to  that  found  in  previous  research  (Rieber,  in  press; 

Rieber  et  al.,  in  press).  Previous  research  found  that  subjects  interacted  far  less  when  given  visual  feedback  even  though 
their  game  scores  were  better.  This  difference  is  probably  due  to  the  simpler  nature  of  the  simulation  and  backs  up  the 
hypothesis  that  subjects  often  increase  their  level  of  overt  interactivity  when  disoriented  (users  seem  to  revert  to  "frantic 
clicking").  The  fact  that  subjects  reported  being  more  frustrated  when  given  verbal  instead  of  visual  feedback  is 
consistent  with  our  previous  research.  Visual  feedback  is  much  preferred  by  subjects  given  such  physics  simulations. 

The  relationship  between  imagery  and  motivation  is  also  consistent  with  dual  coding  theory  (Clark  & Paivio,  1991).  Of 
course,  the  qualitative  data  suggesting  that  some  (but  not  all)  subjects  eventually  switch  their  preference  to  verbal 
feedback  to  match  certain  strategies  they  are  constructing  attests  to  the  need  to  design  simulations  with  a range  of 
feedback  options. 

Several  questions  for  future  research  remain.  More  research  is  needed  on  the  effects  of  the  visual  and  verbal 
elements  of  the  embedded  elaborations  so  as  to  better  understand  how  they  interact  with  different  feedback  representations 
in  the  simulation.  Also,  more  precise  information  on  the  contribution  of  the  embedded  elaborations  versus  the 
simulation  experience  is  needed.  That  is,  the  issue  of  how  much  learning  is  taking  place  just  by  having  subjects  view 
the  elaborations  without  participating  in  the  simulation  is  open  to  question.  Future  research  should  also  expand  on  the 
delicate  demands  of  optimizing  challenge  to  maintain  motivation  in  game-like  activities  embedded  in  a simulation  (see 
Lepper  & Malone,  1987)  and  the  use  of  certain  games  as  measures  of  tacit  knowledge. 

In  conclusion,  the  results  of  this  study  point  to  a simple  yet  powerful  means  of  facilitating  referential 
processing  in  a simulation.  The  use  of  simple  elaborations  of  the  content  modeled  by  a simulation  coupled  with 
appropriate  matching  of  the  simulation's  feedback  to  the  task  appears  to  be  a surprisingly  effective  way  to  guide  students 
to  focus  on  the  most  important  principles  in  the  simulation  without  subverting  personal  discovery  or  risking  decreased 
motivation.  The  qualitative  data,  however,  tempers  this  enthusiasm  by  reminding  us  of  the  complex  way  a user 
perceives  value  and  relevance  to  these  elaborations.  Users  may  feel  reassured  that  additional  guidance  is  available,  but  yet 
may  not  choose  to  consult  such  help  at  appropriate  times  if  they  fail  to  see  the  connection  of  how  embedded  elaborations 
will  lead  to  greater  success  in  completing  a simulation  or  game. 
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Images  surround  us,  overwhelm  us,  influence,  entertain,  inform,  warn,  direct,  delight,  and  enlighten  us.  While 
academia  is  a verbal  enterprise,  our  world  is  a visual,  sensory  one.  Those  of  us  interested  and  involved  with  visual 
literacy  are  dedicated  to  helping  society  recognize  the  power  of  visuals  and  produce  more  meaningful  visual  displays. 
Courses,  texts,  and  conference  sessions  help  students  become  more  visually  “literate”,  able  to  communicate,  understand, 
and  produce  communication  visually. 

The  introduction  of  affordable  multimedia  computers  with  CD-ROM  capacity,  VCR's,  and  connections  to  the 
Internet  and  the  World  Wide  Web  have  created  an  even  greater  opportunity  to  help  society  develop  visual  literacy.  These 
technologies  are  a rich,  varied,  and  inexpensive  source  for  an  almost  infinite  variety  of  images,  including  but  not  limited 
to  graphics,  photographs,  scientific  displays,  computer-generated  art,  fine  art  reproductions,  instructional  graphics, 
consumer  produced  images,  click  art,  and  so  on. 

In  addition  to  its  ease  and  low  cost,  the  World  Wide  Web  has  increasing  popularity  as  one  of  its  main 
characteristics.  Businesses  and  schools  are  creating  their  own  home  pages,  and  “everyone”  wants  to  be  “on  the  Web”. 
Given  all  this,  we  felt  a responsibility  to  help  society  and  our  students  with  the  visual  issues  related  to  the  World  Wide 
Web.  We  added  to  a course  taught  in  our  Instructional  Technology  program  on  Visual  Literacy,  and  designed  the  course 
to  enable  students  to  use  the  WWW  as  a resource  for  their  assignments  and  activities.  In  this  paper  we  provide  the 
following: 

1.  Introduction  and  background  to  using  the  World  Wide  Web  images  in  teaching  visual  literacy; 

2.  A brief  historical  overview  to  images  in  education; 

3.  An  overview  of  one  approach  to  analyzing  visuals;  and 

4.  Locations  of  a variety  of  useful  images  found  on  the  Web  which  may  be  especially  helpful  for 

others. 

Teaching  Visual  Literacy 

Many  courses  in  visual  literacy  exist  at  universities  across  the  country.  Often  these  courses  are  directed  at  art 
students,  education  majors,  and  in-service  teachers,  who  work  with  the  ideas  in  developing  their  own  artistic  creations  or 
teaching  materials.  The  course  we  taught  is  in  instructional  technology,  and  is  directed  at  students  who  would  then  apply 
the  concepts  to  creating  either  student  materials  (instructional  design)  or  teaching  materials  for  any  level  student.  The 
course  has  at  its  goal  the  overall  improvement  of  the  students’  visual  literacy,  especially  their  ability  to  use  and  interpret 
visual  information  and  messages  for  communication.  The  assignments  in  the  course  include  finding,  selecting  and 
analyzing  many  different  visuals  from  a variety  of  sources,  including  click  art,  CD-ROMs,  film,  video  and  television, 
and  the  World  Wide  Web.  Students  also  analyzed  and  created  visuals  intended  to  inform  and  persuade  viewers,  such  as  ads 
and  posters.  As  this  was  an  introductory  course,  use  of  the  computer  resources  and  technologies  was  limited  by 
laboratory  availability;  students  needed  to  work  in  a learning  center  or  other  location  outside  of  class  time.  The 
assignment  to  use  the  WWW  was  very  well  received  by  students;  they  are  as  noted  above  very  excited  about  learning  this 
latest  technology.  In  introducing  the  variety  of  visual  resources  now  available,  we  also  reviewed  for  students  the  history 
of  the  visual  education  movement  in  the  US  as  it  relates  to  the  teaching  of  Visual  Literacy  now. 


Historical  Overview 

An  early  session  of  the  course  focused  on  the  beginnings  of  visual  literacy,  the  visual  education  movement  in 
the  US.  One  perspective  we  shared  involved  the  changing  role  of  the  media  specialist,  or  visual  curator,  as  our  field 
developed  from  visuals  in  collections  and  became  more  involved  with  their  use  in  education.  For  example,  the  museum 
as  an  institution  in  the  United  States  has  a long  standing  tradition.  Our  first  museum  was  established  by  Charles  Wilson 
Peale  and  is  documented  by  his  1 822  self-portrait  The  Artist  in  his  Museum . The  earliest  examples  of  the  visual 
instruction  movement  are  demonstrated  by  the  development  of  museum/school  interaction.  The  end  of  the  nineteenth 
century  and  the  beginning  of  the  twentieth  was  a time  of  extraordinary  growth  for  United  States  museums.  This  fact  is 
evidenced  by  the  following  list  of  museums:  in  1880  the  Metropolitan  Museum  of  Art  was  opened  in  New  York  City;  in 
1905  the  St.  Louis  Museum  was  established  with  strong  links  to  the  school  system;  the  Cleveland  Museum  of  Art  was 
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opened  in  1909;  and  Chicago's  Field  Museum  opened  in  1911,  to  name  just  a few  of  the  major  institutions.  The  types 
of  student  experiences  established  by  the  museum/school  interactions  became  the  basis  for  Hoban's  1937  book 
Visualizing  the  Curriculum.  His  theory  explains  why  these  experiences  were  important.  Essentially,  Hoban  states  that 
the  first  level  of  learning  occurs  when  there  is  interaction  with  the  actual  object.  This  type  of  interaction  has  also  been 
the  most  important  when  dealing  with  a work  of  art. 

Another  important  aspect  of  the  Visual  Instruction  Movement  was  the  use  of  slides  and  film  in  education.  As 
early  as  1895,  Chicago  school  principals  realized  the  importance  of  visual  instruction.  They  used  personal  funds  to 
establish  a slide  collection  for  their  schools'  use.  By  1917  they  had  amassed  over  8,000  slides  and  donated  the  collection 
to  the  School  Board  for  district  wide  circulation.  Also  by  this  time  there  was  a visual  instruction  course  included  in 
teacher  education  preparation  at  the  University  of  Minnesota. 

Effects  of  Specialization 

It  would  appear  that  visual  instruction  was  becoming  an  integral  part  of  the  nations'  education.  However,  with 
the  introduction  of  various  media,  slides  and  film,  there  developed  the  specialization  in  production  and  administration  of 
materials.  This  technology  specialization  separated  the  visual  instruction  component  from  classroom  delivery.  A new 
territory  was  created,  that  of  the  specialist.  Visual  instruction  ownership  moved  from  the  teachers  to  the  specialist.  The 
scenario  that  appeared  was  one  of  the  teacher  having  to  go  to  the  specialist  for  the  images.  (This  presentation  was 
illustrated  by  art  images  from  the  Art  Department  collection).  An  art  image  that  comes  to  mind  is  that  of  Ingres'  Jupiter 
and  Thetis.  In  this  nineteenth  century  painting  Thetis  is  clearly  begging  Jupiter  for  favors.  In  the  past,  libraries  or  visual 
resource  centers  often  guarded  their  materials  rather  than  share  them.  The  specialist  could  wield  power  over  the 
distribution  of  images  much  like  the  power  displayed  in  this  image  of  Jupiter.  This  isolationist  philosophy  of  some 
specialists  may  have  lead  in  part  to  the  decline  of  the  visual  instruction  movement. 

According  to  Saettler  (1990),  the  decline  of  the  visual  instruction  movement  occurred  after  World  War  II.  Then 
current  communication  models  explored  the  concrete-abstract  continuum  first  outlined  by  Hoban  (1937)  and  later  defined 
by  Dale's  Cone  of  Experience.  However,  these  models  were  not  reflected  in  the  specialists'  philosophy.  These  first 
Instructional  Technologists  had  become  a small,  isolated  group  of  people  who  enjoyed  top-level  decision  making 
positions  in  regard  to  administrative  functions  and  the  production  of  visual  instruction,  but  who  ignored  current,  relevant 
psychological  theories.  This  decline  could  be  illustrated  by  Thomas  Couture's  Romans  of  Decadence.  Like  the  Roman  s 
who  lived  only  in  Rome,  isolating  themselves  from  the  outlying  areas,  the  visual  specialists  took  on  a role  of  authority 
and  power  separating  themselves  from  the  educators. 

The  paradigm  shift  that  occurred  in  the  1970s  from  behaviorism  to  cognition  provided  a new  role  for  visual 
instruction.  Olson  and  Bruner  stated  that,  "instructional  media,  therefore  cannot  be  chosen  simply  in  terms  of  their 
ability  to  convey  certain  kinds  of  content,  but  also  be  chosen  in  terms  of  their  ability  to  develop  the  processing  skills 
that  make  up  such  an  important  part  of  human  intelligence”  (Saettler,  1990,  p.440).  The  focus  then  in  this  new 
paradigm  shifted  from  the  media  like  film  (i.e.  what  is  the  film  doing  to  the  learner)  to  the  thought  processes  involved  in 
evaluating  that  film  or  image.  Depth  of  processing  could  then  be  used  to  evaluate  images;  consequently,  the  technology 
or  medium  c/should  become  transparent  to  a point.  This  paradigm  shift  breathed  new  life  into  the  visual  instruction 
movement,  which  has  been  renamed  visual  literacy  since  1969. 

Analyzing  Visual  Messages-A  Construct 

Visual  literacy  research  can  provide  a structural  framework  for  courses  such  as  Visual  Literacy  or  Introduction  to 
the  Visual  Arts.  Barbara  Fredette's  research  combined  with  the  theories  of  Amheim  and  Feldman  create  her  approach  to 
reading  pictures  or  images  (Fredette,  1994).  Her  method  demonstrates  a depth  of  thought  processing  which  can  be  applied 
to  this  course.  Fredette  describes  the  process  in  four  steps;  description,  analysis  of  form,  creative  interpretation  and 
critical  interpretation.  She  states  that  based  on  her  research  with  students,  viewers  go  through  each  of  these  processes  in 
the  order  stated  and  that  this  order  represents  an  hierarchy  of  thought  processing. 

It  has  been  our  experience  with  college  level  students  over  the  past  five  years  that  they  process  a work  of  art  in  a 
slightly  different  manner.  They  use  the  same  four  steps  but  process  them  in  a slightly  different  order;  description, 
creative  interpretation,  analysis  of  from  and  critical  interpretation.  We  believe  they  change  this  order  for  the  following 
reasons.  Students  have  the  ability  to  provide  a description  and  a creative  interpretation  without  any  additional 
information.  They  can  describe  images,  shapes,  color  and  lines.  They  also  readily  apply  their  own  knowledge  to  the  art 
work  as  they  attempt  to  'make  sense'  of  it  within  their  own  experiences.  However,  analysis  of  form  demands  that  they 
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know/leam  the  terminology  of  analysis  and  that  they  can  apply  those  terms  to  works  previously  unseen.  This  process 
develops  critical  thinking  skills.  Secondly,  they  must  bring  with  them  some  research  skills  for  the  fourth  level , the 
critical  interpretation  of  a work.  Viewers  must  find  the  resources  necessary  to  provide  themselves  with  additional 
information,  such  as  culture,  iconography  and  relationship  to  the  media.  Hopefully  they  will  become  more  informed 
viewers  about  the  work  of  art. 

In  conclusion,  Fredette's  framework,  modified,  provides  a solid  structure  for  the  students  in  introductory  visual 
arts  courses.  The  students  need  to  understand  which  level  of  explication  is  currently  being  used  in  their  classroom  setting, 
and  need  to  be  able  to  work  between  levels  to  help  themselves  fully  understand  a visual  work  and  expand  their  visual 
literacy. 

Using  the  World  Wide  Web 

Many  Web  sites  have  been  located  which  could  be  shared  with  students  and  used  for  visual  examples  in  classes. 
Our  students  drew  from  these  sources  and  others  to  analyze  and  present  their  ideas  in  class.  Since  the  Web  is  ever- 
changing,  the  list  presented  is  of  course  dated  as  we  print  it  out.  But  these  sources  should  help  lead  to  others  current  at 
the  time  of  use.  (Examples  from  the  following  sites  were  shared  using  slides  copied  from  files  taken  from  the  Web). 

Art  related  Websites  (fall,  1996) 


World  Wide  Arts  Resource 
http://www.concourse.com/wwar/default.html 

ArtServe,  Australian  National  University. 
http://rubens.anu.edu.au 

AIDS  Memorial  Quilt  Website 
http://www.aidsquilt.org/ 

Ansel  Adams 

http://www.book.uci.edu/AdamsHome.html 
Asian  Arts 

http://www.webart.com/asianart/index.html 

Christo's  and  Jeanne-Claude’s  homepage 
http://pomo.nbn.com/youcan/christo/index.htm 

Christus  Rex  the  Vatican  museums 
http://www.christusrex.org/ 

Frida  Kahlo 

http://www.cascade.net/kahlo.html 

Indianapolis  Museum  of  Art 
http://www.otisnet.com/ima 

Institute  of  Egyptian  Art  and  Archaelogy 
http://www  .memst.edu/egypt/main.html 

International  Visual  Literacy  Association 
http://www.emporia.edU/S/www/slim/resource/IVLA/IVLA.htm 

Le  WebMuseum 
http://www.emf.net/wm/ 
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Minneapolis  Institute  of  Arts 
http://www.mtn.0rg/MIA/miaJntr0.html#index. 


Museum  of  Bad  Art 
http://glyphs.com/moba/ 

National  Museum  of  African  Art 

http://www.si.edu/organiza/museums/africart/start.htm 
National  Museum  of  Slovenia 

http://stenar.ames.si/guest/ljnarodnim4/ang/index.html 
On-Line  Visual  Literacy  Project 

http://www.pomonaxlaremont.edu/visual-lit/intro.html 

Oriental  Institute  Museum 
http://www-oi.uchicago.edu/ 

Prehistoric  caves  at  Vallon-Pont-d’Arc 
http://www.culture.fr/culture/gvpda-en.htm. 

Surrealism 

http://pharmdec.wustl.edu/juju/surr/suiTealism.html 

Teaching  Contemporary  Art:Sandy  Skoglund 
http:/ w w w . artsednet  .getty . ed  u/ 


Andy  Warhol 

http://www.warhol.org/warhol/warhol.html 

Women  Artists  Archive 
http://www.sonoma.edu/library/waa/ 

World  Art  Treasures 

http://sgwww.epfl.ch/BERGER/index.html 

Summary 

Art  images  are  a natural  choice  for  the  teaching  of  visual  literacy.  Many  academic  settings  have  art  collections 
available,  but  students  do  not  always  have  such  access.  The  history  of  the  visual  instruction  movement  may  not  be 
familiar  to  many  students  in  educational  technology,  who  may  think  that  visuals  were  newly  introduced  with  television 
and  computer  software.  The  historical  perspective  summarized  here  is  one  that  helps  students  understand  the  importance 
and  acceptance  of  the  visual  instruction  perspective.  In  addition,  the  use  of  new  technologies  makes  the  teaching  of 
visual  literacy  more  current  and  viable  for  instructional  technology  students.  It  is  hoped  that  the  background,  analysis 
techniques,  and  Web  sites  included  here  make  the  teaching  of  visual  literacy  more  interesting  and  valuable  for  others. 
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Introduction 

The  current  uproar  that  is  the  field  of  education  in  the  1990's  seems  to  place  almost  every  aspect  of  teaching  and 
learning  into  question.  The  battles  that  stem  from  major  philosophic  differences  tend  to  permeate  not  only  educational 
philosophies  and  research,  but  curriculum  development  and  instructional  practice  as  well.  In  instructional  design,  such 
philosophic  debate  has  manifested  as  a tension  between  instructivist  and  constructivist  approaches.  Designing 
instruction,  in  this  environment,  becomes  a statement  of  the  underlying  assumptions  about  knowledge  and  learning  held 
by  the  designer  and  the  client,  as  well  as  a definition  of  the  roles  of  the  teacher  and  students  who  experience  the 
instruction. 

Even  though  the  instructivist  and  constructivist  instructional  design  approaches  appear  to  be  at  odds,  attempts 
have  been  made  to  many  the  two: 

For  ID  [instructional  design],  this  means  primarily  re-integrating  design  with  instructional  delivery  so 
that  the  interaction  between  student  and  instruction  becomes  more  adaptive  to  situations  that  vary  from  one 
problem  to  the  next.  It  also  means  the  development  of  learning  environments  in  which  students  construct 
knowledge  for  themselves.  For  their  part,  proponents  of  SL  [situated  learning]  need  to  develop  effective  ways 
for  bringing  authentic  activity  into  the  classroom....  I expect  that  with  a serious  attempt  to  adapt  ID  to  SL  and 
vice  versa,  powerful  and  innovative  learning  environments  can  be  built  to  the  benefit  of  all  who  work  in  them 
(Winn,  1993,  p.  20). 

At  the  1995  AECT  National  Convention,  three  different  presentations  on  constructivist  instructional  design 
were  offered  which  attempted  to  answer  Winn's  call  for  "innovative  learning  environments"  and  the  merger  of  instructivist 
and  constructivist  design:  (a)  the  "Layers  of  Negotiation  Model"  (Cennamo,  Abell,  & Chung,  1995),  (b)  a design  for 
graduate  school  courses  (Wager,  Lebow,  Driscoll,  & Supinski,  1995),  and  (c)  a design  for  a series  of  business  courses 
(Cognitive  Learning  Group,  1995).  Though  all  three  presentations  were  clearly  based  on  both  constructivist  theory  and 
on  recommendations  from  the  literature,  such  as  Driscoll's  (1994)  five  conditions  for  constructivist  learning 
environments,  no  attempt  was  made  to  present  a visual  model.  Cennamo  et  al.,  (1995)  came  closest  when  they  described 
the  Layers  of  Negotiation  Model  as  a spiral,  yet  they  did  not  offer  a graphic  representation  of  the  spiral  process. 

While  visual  or  graphic  models  in  instructional  design  that  follow  an  instructivist  philosophy  are  readily 
available  (Dick  & Carey,  1990;  Gagn6,  Briggs,  & Wager,  1992),  participants  in  the  philosophical  debates,  theoretical 
discussions,  and  design  efforts  in  constructivist  learning  environments  have  not,  to  our  knowledge,  produced  a visual 
representation  of  constructivist  instructional  design.  One  might,  at  first  blush,  question  the  need  for  another  instructional 
design  model.  Yet,  when  we  were  faced  with  designing  and  developing  our  own  new  courses  following  a constructivist 
philosophy;  we  found  that  a model,  visual  representation,  or  "graphic"  that  incorporated  both  a systematic  design  process 
and  a constructivist  approach  could  provide  a useful  framework  for  efficient  design. 

In  this  paper,  we  present  our  design  processes  for  two  different  learning  contexts.  These  processes  are  based  in 
part  on  the  inspiration  gained  from  the  presentations  of  Cennamo  et  al.,  Wager  et  al.,  and  the  Cognitive  Learning  Group 
at  the  1995  AECT  convention.  We  also  draw  upon  instructional  design  research  and  on  the  larger  community  of 
educational  research  to  (a)  present  a three-part  design  framework  based  on  the  core  instructional  models  first  proposed  by 
Sylvia  Famham-Diggory  (1994)  and  the  stages  of  learning  described  by  Jonassen  (1991a),  (b)  examine  the  framework's 
integrity  and  usefulness  by  asking  five  questions  first  posed  by  Dick  and  Carey  (1990)  and  explored  by  Dunn  in  his 
review  of  constructivist  philosophy  (Dunn,  1994) , and  (c)  briefly  describe  how  the  framework  answers  questions  that 
concern  learning  raised  in  construed  vist/instructivist  discussions  (Driscoll,  1994;  Jonassen,  1991b). 

We  must  hasten  to  state  that  we  are  not  interested  in  prescribing  a series  of  events  or  designing  from  discrete 
stages  (Cennamo  et  al.,  1995)  in  our  presentation  of  a constructivist  model  of  instructional  design.  Instead,  we  approach 
a constructivist  model  of  instructional  design  both  from  the  design  perspective  and  from  the  learner's  context.  In 
considering  constructivist  instructional  design,  or  more  accurately,  instructional  context  design , the  larger  contexts  of 
designer,  teacher,  and  learner  are  incorporated  into  a framework  that  supports  a context-sensitive  teaching/leaming 
environment  rather  than  a prescriptive  recipe  for  delivering  content. 

This  concept  of  our  visual  representation  as  a framework  rather  than  a prescriptive  model  for  constructivist 
educational  design  is  important.  Rather  than  establish  a specific  set  of  steps,  we  envision  a supportive  structure , within 
which  constructivist  design  might  take  place.  This  supportive  structure  is  similar  in  spirit  to  Constructivist  art  of  the 
1920's  with  its  "emphasis  upon  the  structural  aspect  of  the  work  of  art  [which]  led  to  the  concept  of  constructed  art" 
(Hamilton,  1972,p.  235).  Similarly  in  constructivist  instructional  design,  the  design  framework  with  its  focus  on  the 
context  of  learning,  promotes  the  construction  of  learning  within  each  context. 
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Designing  Constructivist  Instruction 


Origins  of  the  Concept 

Reflecting  on  our  past  instructional  design  projects,  we  recalled  that  the  design  models  which  had  been  most 
helpful  to  us  were  those  which  provided  a direct  connection  between  design  steps  and  an  organized  approach  to  learning 
theory.  The  “Stages  in  Designing  Instructional  Systems,”  as  integrated  with  Gagne’s  “Events  of  Instruction”  (Gagn6,  et 
al.,  1992)  was  favored  by  one  of  us  in  early  instructional  design  projects.  The  other  used  Tennyson  and  Rasch’s 
“Instructional  Planning  Model,”  which  offered  “a  model  of  instructional  design  that  links  various  forms  of  cognitive 
learning  to  appropriate  methods  of  instruction”  (Tennyson  & Rasch,  1988,  p.  369). 

While  the  experiences  reported  by  Cennamo  et  al.,  Wager  et  al.,  and  the  Cognitive  Learning  Group  made  it  clear 
that  constructivist  design  could  not  be  reduced  to  a linear  series  of  prescribed  steps,  we  approached  our  consideration  of  a 
framework  for  instructional  context  design  with  the  belief  that,  even  in  a non-linear  design  process,  a conceptual  link  to 
learning  theory  remained  vital.  However,  taxonomies  or  lists  of  learning  conditions  seemed  more  congruent  with  linear 
design  models  than  with  the  spiraling  or  layered  process  of  constructivist  design  that  the  various  presenters  at  the  1995 
AECT  convention  described. 

Therefore,  we  sought  a non-hierarchical  conceptual  model  of  learning  theory  which  would  be  simple,  inclusive, 
and  flexible.  We  found  that  combination  of  characteristics  in  the  three  core  instructional  models  (linked  to  five  types  of 
acquired  knowledge)  described  and  delimited  by  Sylvia  Famham-Diggory  in  her  1994  analysis  of  educational  research 
literature.  Famham-Diggory  maintains  that: 

Every  instructional  program  ...  can  be  described  as  fitting  one  of  three  mutually  exclusive  models  ... 

behavior,  development,  or  apprenticeship. ...  [T]he  nature  of  the  model  is  determined  by  two  factors:  how  the 

model  distinguishes  novices  from  experts  and  what  the  mechanism  of  transformation  is  (1994,  p.  464). 

In  the  behavior  model,  novices  are  distinguished  from  experts  by  quantitative  differences  on  the  same  scale.  The 
mechanism  of  transformation  is  what  Famham-Diggory  terms  “incrementation,”  getting  better,  faster,  or  going  higher  on 
a quantitative  scale.  The  development  model  distinguishes  between  experts  and  novices  by  differences  in  qualitative 
measures,  such  as  personal  beliefs  and  attitudes.  The  change  is  accomplished  through  “perturbation,”  the  challenging  of 
personal  theory. 

In  the  apprenticeship  model,  “experts  and  novices  are  from  different  worlds”  (p.  466).  The  transformative 
process  is  “acculturation,”  that  is,  one  learns  about  expertise  by  living  in  the  culture  of  the  expert.  Famham-Diggory 
maintains  that  within  each  of  these  three  instructional  models,  any  of  five  types  of  knowledge  can  be  acquired: 
declarative,  procedural,  conceptual,  analogical,  and  logical 

As  we  began  to  create  our  visual  representation  of  constructivist  instructional  design,  we  were  guided  by  David 
Jonassen’s  assertion  that  designers  cannot  always  control  what  individuals  learn,  but  that  they  can  aspire  to  design 
environments  which  support  learners  in  their  learning  (Jonassen,  1991a).  We  adopted  his  concept  of  three  stages  of 
knowledge  acquisition  (initial,  advanced,  expertise)  (Jonassen,  1991b),  each  entailing  different  types  of  learning 
(practice/feedback  for  the  initial  stage  of  learning,  apprenticeship/coaching  for  the  advanced  stage,  experience  for  the 
expert  stage),  the  whole  forming  a continuum  of  learning  and  experience.  These  stages,  used  in  conjunction  with  the 
Famham-Diggory  instructional  models,  illustrate  the  multiple  combinations  of  learner  characteristics,  instructional 
approaches,  and  content  types  which  can  interact  in  an  instructional  context,  and  which  must  be  addressed  in  the  design 
process. 

The  task  for  a constructivist  designer  is  to  create  an  instructional  context  rich  in  experiences  which,  in 
interaction  with  the  learner,  may  provide  for  the  development  of  expertise.  We  use  the  terms  “expert”  and  “novice” 
relatively,  as  does  Famham-Diggory  (1994,  p.  464).  One  could,  for  example,  be  expert  at  recognizing  the  upper  case 
“A”,  or  be  a novice  at  diagnosing  a particular  variety  of  influenza.  The  instructional  context  designed  for  learners  in  each 
situation  might  draw  on  a combination  of  the  instructional  models  in  creating  an  environment  supportive  of  the  learner’s 
development  within  that  content  domain. 

While  Famham-Diggory  defines  the  three  instructional  models  as  mutually  exclusive,  she  recognizes  that  more 
than  one  can  function  concurrently  within  an  instructional  program.  Current  research  (Eisner,  1993;  Salomon,  1991) 
suggests  that  sharp  delimiters  between  such  models  or  levels  are  rare  in  real  learning  contexts.  Within  the  same  content 
domain,  learners  may  be  novices  in  some  elements  and  more  expert  at  other  elements,  yet  overall  be  categorized  at  a 
developmental  stage  within  the  larger  focus  of  the  domain  (two  examples  will  be  discussed  later  in  the  paper).  With  this 
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in  mind,  we  attempted  to  include  in  our  visualization  of  instructional  context  design  some  means  of  illustrating  the  gray 
areas  between  and  among  the  three  core  learning  models  and  the  stages  of  knowledge  acquisition. 


Creating  the  Visualization 

Learning  context,  with  all  of  its  gray  areas,  complexities,  and  uncontrollable  variables,  became  the  focus  of  our 
conceptual  model.  The  challenge  from  Winn  (1993)  to  create  “innovative  learning  environments”  (p.  20),  and  the  process 
of  “spiral  cycles  rather  than  discreet  stages”  described  by  Cennamo  et  al.  (1995,  p.40),  shaped  our  concept  of  an 
instructional  context  as  a circular  "tunnel"  through  which  the  learner  passes  while  experiencing  the  lesson,  course,  or 
instructional  system  (see  Figure  1).  Within  the  tunnel,  all  experiences,  materials,  interactions,  and  communications, 
combine  with  the  learner’s  prior  knowledge  to  create  the  instructional  context  which  is  designed  with  the  intent  of 
stimulating  the  learner  to  construct  knowledge  relevant  to  the  goals  of  the  learning  experience.  As  learners  pass  through 
the  instructional  context  tunnel,  they  will  progress  from  the  novice  level  to  some  level  of  greater  expertise  in  the  relevant 
content  domain.  Thus  the  concept  of  a spiral  tunnel  attempts  to  include  variables  that  enter  into  any  learning  context  as 
well  as  the  intended  content  of  the  learning  experience. 

Viewed  over  time,  that  is,  over  the  length  of  the  course,  semester,  or  learning  experience;  the  tunnel  appears  as 
an  expanded  coil  spring  through  the  center  of  which  the  learner  passes  as  time  passes  in  the  learning  environment  (see 
Figure  2).  The  spiral  bands  represent  the  three  core  instructional  models  as  they  are  cycled  through  repeatedly  by  the 
learner  during  the  process  of  experiencing  the  learning  context  over  time.  The  overall  tunnel  segment  (from  the 
"beginning”  of  the  instruction  to  the  "end"  of  instruction)  represents  the  gradual  progress  through  the  novice,  advanced, 
and  expert  levels  of  learning. 

Figure  1.  A cross-sectional  view  of  the  Instructional  Context  Design  tunnel  illustrating  three  core  instructional 
paradigms  at  any  point  in  time. 


Figure  2,  A side  view  of  the  Instructional  Context  Design  tunnel  illustrating  the  repeating  process  of  moving  through 
novice,  advanced,  and  expert  stages  of  knowledge  acquisition  over  time. 


Having  visualized  the  spiral  tunnel,  we  set  out  to  apply  it  to  the  instructional  design  projects  in  which  we  were 
involved:  the  first  in  a K- 12  setting,  the  second  in  a university  course.  We  did  not  intend  to  abandon  traditional  design 
questions,  but  hoped,  like  Cennamo  et  al.  (1995),  to  shift  the  emphasis  from  procedures  to  process.  Beginning  at  the 
expert/exit  end  of  the  tunnel,  we  anticipated  examining  the  expert  culture  for  that  content  domain,  and  then  working  back 
through  the  spiral  to  the  point  at  which  the  novice  learner  enters  the  instructional  context.  As  we  moved  through  the 
tunnel,  we  hoped  to  follow  the  spiral  as  it  wound  around  the  tunnel  wall,  identifying  the  specific  instructional  models 
most  appropriate  to  the  stage  of  knowledge  acquisition  represented  at  each  point  of  the  process. 

The  next  two  sections  describe  the  experiences  each  of  us  had  in  applying  the  Instructional  Context  Design 
framework  (hereafter  referred  to  as  ICD).  We  offer  these  personal  accounts,  not  as  formal  research  in  instructional  design, 
but  as  real  world  applications  of  the  framework  in  the  hope  of  promoting  further  discussion  of,  and  reflection  on,  the 
process  of  applying  a constructivist  approach  to  the  design  process. 

Design  into  Practice  in  a K-12  Setting 

As  an  instructional  consultant  in  an  elementary  school,  I am  frequently  involved  with  teachers  in  developing 
lessons  and  units  of  instruction.  A teacher  will  often  approach  me  with  an  instructional  problem,  and  we  will  collaborate 
in  creating  materials  or  planning  learning  activities  to  address  the  problem.  Since  first  visualizing  the  spiral  tunnel 
approach  to  ICD,  I have  had  several  opportunities  to  apply  this  process  in  design  projects  with  teachers. 

In  one  case,  a fifth  grade  teacher  approached  me  with  a problem  in  the  reading  and  language  arts  curriculum 
areas.  The  district  curriculum  required  that  a specific  research  process  be  taught,  and  that  an  example  of  research  writing 
be  included  in  each  student’s  portfolio.  The  teacher  reported  that  the  research  process  took  up  too  much  instructional 
time,  that  the  student  writing  was  of  poor  quality,  and  that  most  students,  even  those  who  generally  enjoyed  writing, 
disliked  the  research  writing  process.  The  challenge  was  to  develop  an  instructional  unit  which  could  be  completed  by 
students  in  about  two  weeks,  which  would  include  the  processes  and  products  required  by  the  district  curriculum  guide, 
and  which  would  be  motivating  for  more  of  the  students. 

We  began  our  design  process  by  visualizing  the  exit  of  the  tunnel,  exploring  together  the  components  of  the 
culture  of  the  expert  research  writer  at  this  level.  We  consulted  curriculum  guides,  student  research  writing  from  past 
classes,  fifth  grade  student  writers,  and  the  experiences  of  several  writing  teachers  as  we  constructed  our  image  of  the 
knowledge  base,  behaviors,  attitudes,  and  products  which  might  characterize  a skilled  and  competent  research  writer.  As 
we  uncovered  these  characteristics,  we  were  concurrently  engaging  in  a dialogue  about  the  setting  in  which  these  expert 
qualities  would  be  likely  to  flourish  - the  cultural  context  of  the  expert  fifth  grade  research  writer. 

As  we  approached  consensus  on  our  vision  of  the  tunnel  exit  and  the  emerging  expert  research  writer,  we  shifted 
our  focus  toward  visualizing  the  spiraling  tunnel  from  above.  In  this  phase  of  the  process,  we  discussed  what 
combination  of  instructional  models  might  best  support  a learner  moving  through  the  tunnel,  engaging  in  the  process  of 
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repeatedly  cycling  through  stages  of  knowledge  acquisition.  Shifting  our  visualization  to  the  tunnel’s  tri-shaded  walls  as 
viewed  from  its  entrance  reminded  us  that  at  any  point  in  the  instructional  context,  die  learner  might  have  need  for 
behavioral,  developmental,  or  apprenticeship  instruction.  Changing  perspective  again  to  visualize  the  smaller  tri-colored 
bands  spiraling  around  the  coils  of  the  tunnel  wall  helped  us  to  remember  that  at  any  point  in  the  instructional  process 
the  learner  might  be  functioning  as  novice,  advanced  learner,  or  expert,  relative  to  any  particular  element  of  the 
instructional  context.  As  this  phase  of  visualization  proceeded,  I became  very  conscious  of  Cennamo' s comments  about 

the  necessity  of  embracing  the  complexity  of  the  design  process. 

The  case  of  the  fifth  grade  research  writing  project  was  typical  of  my  experiences  using  the  spiral  tunnel 
visualization,  in  that  just  as  all  the  spirals  and  the  considerations  they  represented  began  to  feel  too  confusing,  the 
confusion  crystallized  into  some  obvious  and  relatively  simple  answers.  This  stage,  which  I think  of  as  the  third  and 
final  stage  of  the  visualization  process,  feels  to  me  both  non-linear  and  systematic.  I visualize  it  as  having  completed 
construction  of  a three  dimensional  scale  model  of  the  tunnel  (and  the  instructional  context  it  represents).  At  this  stage, 
we  can  peer  into  the  entrance  and  exit,  and  peek  through  the  spiraling  walls  at  any  point  along  their  course. 

The  following  paragraph  of  instructions  was  the  end  result  of  the  peeking  and  poking  in  the  ICD  tunnel  we 
constructed  for  the  fifth  grade  research  writing  project.  These  directions  were  supported  by  a two-page  form,  some  teacher 
modeling,  a few  reference  materials,  and  some  student  collaboration: 

“You  have  been  hired  as  a writer  by  National  Geographic  World  magazine.  Your  assignment  is  to 
write  a twenty  page  short  story  about  a child  your  age  who  lives  in  one  of  the  fifty  United  States  and  who  has 
an  adventure  involving  a wild  animal.  You  will  soon  receive  your  expense  account  money  and  a round  trip 
plane  ticket  to  the  state  of  your  choice.  Before  you  leave  on  this  writing  assignment,  you  must  provide  the 
magazine  editors  with  a completed  story  proposal  which  must  include  accurate  background  information  on  the 
setting  and  characters  of  your  proposed  story,  and  a summary  of  the  plot.  The  editors  need  your  completed  story 
in  time  to  publish  it  in  the  September  issue,  so  your  plot  summary  must  be  on  our  desk  no  later  than  two 

weeks  from  today.” 

The  instructional  delivery  for  these  materials  differed  greatly  from  the  previous  standard  for  fifth  grade  research 
writing  instruction.  Faced  with  an  engaging  challenge,  students  collaborated  with  their  teacher  and  with  each  other  to 
locate  and  explore  useful  reference  sources.  The  teacher  modeled  the  necessary  information  gathering  and  organizing  skills 
as  she  prepared  her  own  story  proposal,  but  spent  the  majority  of  the  “instructional”  time  collaborating  with  individuals 
and  groups  as  they  attempted  to  refine  their  data  gathering  and  writing  skills . 

Two  weeks  later,  twenty  nine  satisfactorily  completed  story  proposals  sat  on  the  teacher  s desk,  ready  for 
inclusion  in  the  students’  writing  portfolios.  The  students,  however  were  not  willing  to  see  this  project  end.  "hey 
requested. and  obtained  class  time  to  hear  each  of  the  story  proposals.  Several  of  them  used  their  proposals  as  the  basis 
for  the  full  length  short  story  which  they  completed  voluntarily  at  home.  The  word  of  mouth  advertising  for  the  fifth 
grade  project  resulted  in  a sixth  grade  teacher  adapting  the  project  for  her  students. 

Unlike  the  course,  project,  and  curriculum  designs  described  by  Cennamo  et  al..  Wager  et  al.,  and  the  Cognitive 
Learning  Group,  the  design  process  described  in  detail  here  took  only  a few  hours  over  a several  days  to  complete.  At 
first,  I questioned  using  a complex  visualization  process  for  such  small-scale  design  projects.  However,  like  Cennamo  et 
al.and  Wager  et  al,  I observed  that  the  type  of  process  used  influenced  the  type  of  product  created.  In  this  elementary 
school  setting,  the  teachers’  approaches  to  instructional  delivery  appeared  to  change  as  they  participated  in  the  ICD 

process. 

The  structured  collaborative  design  effort,  though  brief  and  informal,  combined  with  a non-linear  process  of 
context  design,  consistently  resulted  in  instruction  which  was  perceived  as  more  authentic,  engaging,  collaborative,  and 
stimulating  for  students  and  teachers  than  traditional  classroom  instruction.  Incorporating  the  two  categorization  systems 
- core  instructional  models  and  stages  of  knowledge  acquisition  - resulted  instructional  activities  in  which  students 
evidenced  academic  growth  while  producing  products  of  high  quality. 

Design  into  Practice  in  Higher  Education 

“Art  Media  Techniques:  Computers”  is  a required  course  offered  in  the  post-baccalaureate  program  in  Art 
Education  at  the  university.  The  offer  to  teach  this  course  became  an  opportunity  to  redesign  its  content  and  approach. 
Most  courses  do  not  require  learners  to  encounter  content  from  two  seemingly  unrelated  perspectives,  yet  the  use  of 
computers  in  visual  arts  education  demands  that  both  instructional  uses  and  creative  expression  be  addressed,  often 
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simultaneously.  Since  the  experience  of  producing  art  with  any  media  is  immediate,  authentic,  and  most  often  happens 
in  real  time  (Perkins,  1994)  a constructivist  approach  to  using  the  computer  as  an  art  medium  seemed  pretty 
straightforward  and  definitely  appropriate  for  the  course.  Teaching  with  and  about  computer  technologies  as  an 
instructional  medium  for  other  aspects  of  art  education  from  a constructivist  philosophy  (such  as  aesthetics,  art  history, 
or  art  criticism),  was  a taller  order.  Yet  the  field  of  art  education  and  the  larger  art  world  has  been  dealing  with  the 
philosophical  struggle  between  modernism  and  postmodernism  since  the  earliest  emergence  of  such  writings.  I held 
firmly  to  the  idea  that  if  any  domain  could  be  taught  from  a constructivist  perspective,  the  most  successful  and  clearly 
appropriate  domain  would  be  the  field  of  art  education. 

Working  from  my  experience  in  instructional  design  and  as  an  art  educator,  my  approach  to  the  design  of  this 
course  was  to  first  consider  the  learners  and  the  level  of  expertise  they  might  be  able  to  attain  in  the  course.  Like  Wager 
and  Lebow  (1995),  I spent  time  gathering  information  about  the  previous  offerings  of  the  course  and  about  the  students' 
entry  level  skills  through  interviews  with  other  instructors  and  a few  of  the  students  who  intended  to  take  my  course. 
Based  on  this  information,  I considered  what  these  students  needed  and  expected  from  a course  in  art  education  dealing 
with  computer-based  technology  as  both  an  instructional  medium  and  an  art  medium. 

From  the  information  I had  gathered,  I first  listed  the  strengths  and  constraints  within  which  the  course  would  be 
designed,  much  as  in  a traditional  design  process.  In  this  case,  however,  rather  than  focusing  on  entry  level  skills  needed 
to  complete  predetermined  goals,  the  analysis  was  focused  around  Jonassen's  three  levels  of  knowledge  acquisition  within 
the  learning  context  and  their  potential  interactions  with  Famham-Diggory's  models  of  learning.  The  results  of  this 
analysis  were: 

Strengths 

• a wide  range  of  student  skill  and  experience  levels  in  art  production 

• a wide  range  of  student  skill  and  experience  levels  with  computer  technology 

• an  existing  collaborative  network  among  the  students  enrolled  for  the  course 

• availability  of  adequate  computer  technology  in  the  classroom  (PowerMacs,  direct  Internet  connections, 

classroom  LAN,  application  software) 

Constraints 

• short  duration  (the  class  met  3.5  hours  every  day  for  three  weeks) 

• limited  software  availability  (unable  to  get  certain  site  licenses) 

• delayed  installation  of  promised  peripheral  hardware 

I determined  that  the  level  of  student  expertise  envisioned  was  actually  a range  of  outcomes  that  could  not 
necessarily  be  quantified.  For  some  students,  expertise  might  mean  moving  from  a familiarization  level  to  some  level  of 
utilization  or  integration  of  computer  technologies  (Hooper  & Rieber,  in  press;  Rieber  & Welliver,  1989)  in  their  art 
making  or  teaching.  For  others,  expertise  might  mean  that  they  have  lost  a fear  of  using  a computer  or  have  discovered  a 
previously  unexplored  use  for  computers  in  art. 

Thus,  the  need  for  a rich  learning  context  that  could  address  the  disparate  levels  of  prior  knowledge,  and  the  dual 
role  of  computers  in  art,  was  apparent.  Ultimately,  I synthesized  the  purpose  of  the  course  into  two  emphases: 

1 . to  familiarize  students  with  computer-based  technologies  as  a tool  for  teaching  and  learning  (instructional 

media)  and 

2.  to  introduce  students  to  (or  extend  their  use  of)  computer-based  technologies  as  art  media. 

The  three  week  course  was  then  organized  around  three  computer-based  projects,  written  reactions  to  three  critical 
issues,  peer  reviews,  and  an  ongoing  expectation  of  in-class  collaborative  group  work.  The  following  paragraphs  sketch 
the  process  through  which  some  of  these  course  requirements  emerged  from  the  ICD  framework,  and  give  examples  of 
how  the  design  functioned  during  the  teaching  of  the  course. 

In  keeping  with  the  constructivist  approach  of  the  ICD  framework,  declarative  and  basic  procedural  instruction 
for  each  activity  were  embedded  within  the  larger  context  of  learning  about  computer-based  technologies  as  instructional 
media  and  art  media.  For  example,  manuals  for  software  use  were  available,  but  no  formal  instruction  was  given  from 
the  manuals.  Rather,  I began  the  course  with  an  exploration  of  the  software  in  the  form  of  a studio  problem  as  a basis 
for  using  the  software  as  an  art  medium.  Since  all  of  the  students  were  able  to  produce  art  works  in  other  media,  I could 
assume  a certain  level  of  expertise  in  art.  That  is,  all  of  the  students  were  at  a rather  sophisticated  level  of  expertise 
through  acculturation  in  the  art  world.  They  knew  the  trends,  terminology,  much  of  the  theory  and  philosophies  of  the 
domain,  as  well  as  certain  art  production  skills.  The  course  was  designed  to  provide  opportunities  and  incentives  for  each 
student  to  transfer  that  knowledge  and  to  build  upon  it  within  the  context  of  the  course,  using  the  new  medium.  The 
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ICD  framework  became  a useful  means  of  visualizing  how  learners  could  move  through  the  learning  context  at  different 
levels  of  expertise  and  experiences. 

This  process  worked  very  well.  In  Project  1 (exploring  the  software),  each  student  began  with  the  same  digitized 
DaVinci  image,  manipulated  it  using  various  tools  with  a variety  of  software  to  produce  a final  image.  During  the  in- 
class  presentations,  images  were  sent  via  the  in-class  network  from  each  presenter’s  computer  to  other  computer  screens 
in  the  room.  Students  were  able  to  ask  each  other  how  a particular  effect  was  achieved,  as  well  as  dialog  about  the 
artistic  aspects  of  the  images,  which  increased  their  computer  skills  incrementally,  as  described  in  Farnham-Diggory’s 
( 1994)  model  of  behavioral  learning.  Thus,  the  novice  computer  user  was  also  the  expert  artist  and  budding  aesthetician, 
consistent  with  the  spiral,  multi-dimensional  nature  of  the  ICD  framework. 

By  project  2,  the  novice  computer  users  became  the  advanced  learners  in  a developmental  stage.  Assumptions 
and  personal  theories  made  during  their  out  of  class  lab  work  were  challenged  (or  perturbed,  as  in  Farnham-Diggory’s 
model  of  developmental  learning)  by  more  advanced  peers  or  by  guest  speakers  and  demonstrations.  The  ability  to 
immediately  try  out  the  “truth”  of  their  personal  theories  or  those  of  the  challengers,  particularly  in  the  art  production 
aspect  of  the  course,  afforded  a rapid  advance  through  the  stages  of  knowledge  acquisition  . 

The  projects  and  small  group  activities  also  sparked  heated  discussions  about  issues  such  as  the  unique  attributes 
of  the  computer  as  an  art  medium,  computer  software  as  mimic  of  other  art  media,  the  ownership  and  originality  of 
images,  gender  issues,  the  political  implications  of  access  to  technology,  and  school  districts’  commitment  to  emerging 
technologies. 

Since  debates  in  the  field  of  art  education  are  currently  focused  on  multiculturalism,  educational  reform,  critical 
theory,  and  whether  or  not  to  employ  a discipline-based  approach;  I assumed  that  these  issues  would  also  surface  during 
this  class.  Therefore,  three  statements  were  given  at  the  beginning  of  the  course  and  students  were  asked  to  reflect  on  the 
implications  of  computer-based  technologies  in  the  arts  within  the  larger  context  of  art  education  and  the  socio-cultural 
milieu.  By  the  end  of  each  week,  students  were  asked  to  write  a brief  reaction  paper  to  that  week’s  statement.  Course 
readings,  projects,  and  guest  speakers  related  to  the  statements  provided  the  rich  context  for  reflecting  on  the  statements. 

By  far  the  best  measure  and  evidence  of  learning  in  the  course,  and  the  success  of  the  instructional  design,  were 
the  reaction  papers.  Early  in  each  week  of  the  course,  some  element  of  the  reaction  statement  for  that  week  would  be 
addressed,  either  through  class  discussions,  activities,  or  a guest  speaker.  Challenging  discussions  often  prompted 
students  to  reconsider  certain  of  their  theories  or  opinions  which,  on  reflection,  proved  to  be  not  as  sound  as  they  had 
previously  thought.  In  this  rich  learning  context,  students  would  listen  to  their  classmates,  read  the  course  readings,  and 
write  their  reactions  based  on  experience  and  reflection.  The  papers  were  a delight  to  read  and  had  much  to  say  about 
authentic  learning  and  about  the  acculturation  process  involved  in  developing  expertise,  though  I have  not  yet  analyzed 
their  content  in  any  formal  sense. 

Student  course  evaluations  were  also  revealing.  Some  students  admitted  to  a new-found  courage  to  using 
computers  in  their  own  teaching,  while  others  moved  toward  a more  integrated  level:  the  computer  was  already  a part  of 
their  working  life  but  they  now  found  it  could  be  easily  incorporated  into  their  art  production.  Others  remarked  that  this 
was  the  First  time  they  had  attended  a course  which  was  based  on  the  assumption  that  they  were  responsible  for  building 
their  own  knowledge,  and  marveled  that  everyone  seemed  to  ’’get  a lot  out  of*  the  experience. 

Course  content  and  teaching  strategies  should  always  be  in  a state  of  change.  Next  year’s  course  will  be  based 
on  our  refined  version  of  the  ICD  framework,  not  because  it  is  a good  cookbook,  but  because  the  framework  is  sensitive 
to  changes  in  the  context  of  learning.  Next  year’s  level  of  expertise  may  be  completely  different  due  to  the  students’  entry 
level,  expectations,  or  area  of  focus.  Since  graduate  students  will  be  allowed  to  take  the  course  for  graduate  credit,  we 
expect  an  exciting  mix  of  ideas  and  abilities,  and  a real  challenge  to  the  usefulness  of  ICD. 

Conclusion 

Our  concept  of  Instructional  Context  Design  worked  well  for  us  as  a framework  for  developing  the  learning 
contexts  briefly  described  above.  This  visual  aid  or  “graphic”  was  a useful  tool  for  designing  and  developing  courses 
within  the  constructivist  philosophy.  However,  for  the  framework  to  have  any  lasting  use  for  us  or  for  others,  we  as 
designers,  researchers,  and  practitioners  always  need  to  ask  questions  that  concern  the  usefulness  and  relevance  of  any  such 
design  and  learning  model. 

As  we  reviewed  our  ICD  framework  and  our  experiences  using  it,  we  referred  to  two  sets  of  recommendations 
which  have  been  proposed  for  evaluating  constructivist  instructional  design  and  design  models:  Dunn’s  (1994)  adaptation 
of  formative  evaluation  questions  first  posed  by  Dick  and  Carey  (1990),  and  Driscoll’s  (1994)  five  conditions  for 


constructivist  learning  environments.  While  it  is  beyond  the  scope  of  this  paper  to  discuss  how  ICD  addresses  all  ten  of 
these  recomendations,  we  will  briefly  examine  a few  selected  points  in  the  following  discussion. 

Dunn  adapted  questions  first  posed  by  Dick  and  Carey  (1990)  in  the  context  of  formative  evaluation  of 
instructivist  instructional  design  models.  Like  Dunn,  we  believe  these  questions  to  be  appropriate  for  evaluating 
instructional  design,  regardless  of  its  over-arching  philosophy. 

Dunn's  modified  questions  for  c instructivist  design  are: 

1 . Are  the  materials  appropriate  to  the  learning  outcome? 

2.  Do  the  materials  include  adequate  instruction  on  the  subordinate  skills? 

3.  Are  the  materials  clear  and  readily  understood  by  representative  members  of  the  target  group? 

4.  What  is  the  motivational  value  of  the  materials?  Do  learners  find  the  materials  relevant  to  their  needs 

and  interests? 

5.  Can  the  materials  be  managed  efficiently  in  the  manner  mediated? 

While  the  informality  of  our  design  process  to  date  leaves  us  with  only  subjective  evidence,  we  are  comfortable 
in  responding  affirmatively  to  all  five  questions.  Looking  back  at  our  design  efforts  similar  to  those  which  Cennamo  et 
al.  (1995)  characterize  as  “process-based”  (p.  40),  we  view  elements  of  that  spiral  process  as  supporting  the  attainment  of 
the  results  called  for  in  Dunn’s  questions.  Instruction  in  subordinate  skills  (question  #2),  for  example,  is  addressed  in 
ICD  through  the  process  of  design  by  spiraling  back  from  expert  culture  (outcomes)  to  novice-level  (entry  skills). 

Insuring  the  appropriateness  of  materials  to  the  learning  process  (question  #1),  is  achieved  by  repeatedly  addressing  the 
three  core  instructional  models  in  design,  and  then  again  in  instructional  delivery.  The  remaining  questions  (3  through  5) 
are  best  answered  through  observation  and  interviews  with  learners  in  a systemic  analysis  of  the  application  of  the  design. 

We  found  that  our  second  set  of  recommendations,  Driscoll’s  five  conditions  for  constructivist  learning 
environments  (Driscoll,  1994),  fit  very  well  with  our  concept  of  designing  spiral  learning  contexts.  Cennamo  et  al. 
(1995)  consciously  adapted  Driscoll's  five  conditions  specifically  to  the  design  process.  However,  in  Instructional 
Context  Design,  the  learning  context  and  the  design  context  reflect  each  other.  We  have  found  that  focusing  attention  on 
the  learning  context  and  the  facilitation  of  knowledge  construction  demands  a corresponding  shift  in  the  design  of 
instruction. 

For  example,  in  item  two  of  her  five  recommendations  for  conditions  of  constructivist  learning,  Driscoll 
proposes  that  "social  negotiations  (become  an]  integral  part  of  learning”  (Driscoll,  1994,  p.  365).  Cennamo  et  al.  (1995) 
proposed  that  social  negotiations  become  ’’an  integral  part  of  designing  the  materials”  (p.  36).  In  the  ICD  process,  social 
negotiations  are  not  only  employed  in  the  spiral  design  process,  but  are  also  incorporated  as  a strategy  for  learning  during 
instruction.  Thus  in  the  ICD  framework  as  we  envision  it,  design  and  application  are  tightly  (if  not  inextricably)  linked, 
and  may  not  exhibit  a clear  distinction  between  (a)  design  complexity  and  the  complexity  of  the  learning  context,  (b) 
social  negotiation  during  design  and  within  the  learning  context,  (c)  examination  of  multiple  perspectives  during  design 
and  within  the  leaning  context,  (d)  reflexivity  during  design  and  during  learning,  and  (e)  client-centered  design  and  student- 
centered  instruction  (Cennamo  et  al.,  1995:  Driscoll,  1994).  We  maintain  that  attention  to  the  context  of  instruction 
during  the  design  process  provides  a critical  and  necessary  interdependence  between  design  processes  and  learning 
processes. 

This  is  not  to  say  that  designing  instruction  for  constructivist  contexts  lacks  structure  or  identifiable  design 
procedures.  Certain  steps,  or  structural  elements,  are  available  for  the  constructivist  instructional  designer.  Rita  Richey 
(1986)  noted  that  most  procedural  models  (as  distinct  from  conceptual  or  mathematical  models)  of  instructional  design 
“...reflect  current  and  proposed  practice.  They  identify  steps,  not  relationships  among  variables;  their  primary  function  is 
to  facilitate  application,  rather  than  to  describe  and  explain  events"  (p.  94).  Such  steps  or  elements  of  procedural 
instructional  design  models  serve  as  a "...source  of  knowledge  upon  which  theories  can  be  constiucted"  (p.  94).  Richey 
synthesized  several  procedural  models  and  identified  six  core  elements  of  systematic  design  processes: 

1 . Determine  learner  needs 

2.  Determine  goals  and  objectives 

3.  Construct  assessment  procedures 

4.  Design  and/or  select  delivery  approaches 

5.  Try  out  the  instructional  system  and 

6.  Install  and  maintain  the  system 

In  the  ICD  framework,  experience  in  terms  of  the  focus  on  learning  contexts,  provides  the  conceptual 
relationship  between  the  "objectivist"  design  steps  and  the  "constructivist"  concern  with  complex  learning  contexts, 
social  negotiations,  multiple  perspectives,  reflexivity,  and  student-centered  instruction  (Driscoll,  1994).  Richey  (1986) 
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states  that  "the  credibility  of  experience  as  a source  of  knowledge  generation  depends  on  the  extent  to  which  these 
experiences  are  viewed  with  objectivity  and  on  how  broadly  they  can  be  applied”  (p.94). 

Reflecting  on  the  spiral  process  involved  in  ICD,  and  on  the  organic  and  participatory  nature  of  the  instruction, 
we  examined  our  learning  contexts  in  terms  of  the  six  elements  described  by  Richey  as  an  attempt  to  "view  with 
objectivity"  the  usefulness  of  the  ICD  framework.  In  ICD,  the  elements,  which  are  evident  in  most  instructional  design, 
do  not  appear  as  linear  and  discrete  steps  in  the  process,  but  rather  as  parts  of  recurring  cycles,  or  as  elements  within  the 
organic  whole  of  the  instructional  context.  Learner  needs  (core  element  #1)  for  example,  are  identified  at  each  stage  of  the 
process  of  tracing  back  from  expert  to  novice  during  the  design  spiral.  They  are  one  of  the  important  issues  encountered 
at  each  level  of  that  analysis.  Later  during  the  instruction,  as  learners  move  through  the  instructional  context,  their  needs 
come  into  play  in  the  interaction  of  prior  knowledge,  instructional  models,  and  levels  of  learning. 

While  we  conceptualize  these  steps  in  ICD  as  dynamic  and  continuous  stages  as  learners  spiral  through  the 
learning  context,  we  agree  that  it  can  be  useful  to  consider  the  stages  and  levels  as  if  they  are  discrete  entities.  From  a 
micro,  or  analytic,  perspective,  such  consideration  can  serve  as  a microscope  for  identifying  the  specific  instructional 
components  involved  in  designing  a course,  lesson,  or  instructional  system.  From  a macro,  or  systemic,  perspective,  the 
levels  and  stages  can  be  helpful  in  providing  a broad  telescopic  view  of  the  complex  constellation  of  elements  comprising 
the  whole  of  an  instructional  endeavor  (Salomon,  1991).  We  believe  this  approach  to  design  and  instruction  results  in  an 
emphasis  on  process  rather  than  steps  or  procedures,  but  still  addresses  the  questions  and  critical  issues  raised  by  Dunn, 
Driscoll,  Richey,  and  others. 

Both  of  our  design  examples  are  small  and  were  first  time  attempts  to  consciously  design  from  a constructivist 
perspective.  Our  materials  were  never  intended  for  mass  production  for  off-the-shelf  curriculum  building.  However,  we 
will  most  likely  use  much  of  what  we  have  designed  and  presented  here  in  future  courses.  We  believe  that  both  attempts 
exhibit  sound  instructional  design  qualities,  planning,  and  procedures.  We  hope  that  the  process  described  here  will  aid 
designers  and  instructors  in  their  efforts  to  develop  effective  means  for  approaching  their  tasks  from  a constructivist 
perspective. 

We  have  concluded  that  attention  to  the  context  of  learning  and  to  how  that  context  facilitates  learning,  is  the 
"value  added"  by  adopting  a constructivist  epistemology  in  instructional  design.  We  do  not  advocate  abandoning  concern 
for  the  delivery  system  or  the  course  content,  but  believe  that  a visual  or  graphic  framework  such  as  Instructional 
Context  Design  may  help  us  avoid  approaching  instruction  and  design  as  if  media,  method,  and  application  of  the  design 
are  separate  entities. 


References 

Cennamo,  K.  S.,  Abell,  S.  K.,  & Chung,  ML-L.  (1995).  A "Layers  of  negotiation"  model  for  designing  constructivist 
learning  materials.  Proceedings  of  selected  research  and  development  presentations  of  the  Association  for 
Educational  Communications  and  Technology,  32-42. 

Cognitive  Learning  Group  (1995).  Toward  a model  far-designing  constructivist  learning  environment.  Paper  presented  at 
the  national  convention  of  the  Association  for  Educational  Communications  and  Technology,  Anaheim,  CA. 

Dick,  W.,  & Carey,  L.  (1990).  The  systematic  design  of  instruction  (3rd  ed.).  Glenview,  IL:  Scott  Foresman. 

Driscoll,  M.  P.  (1994).  Psychology  of  learning  for  instruction.  Boston,  MA:  Allyn  & Bacon. 

Dunn,  T.  (1994).  If  we  can’t  contextualize  it,  should  we  teach  it?  Educational  Technology  Research  and  Development. 
42(3),  83-92. 

Eisner,  E.  W.  (1993).  The  emergence  of  new  paradigms  for  educational  research.  Art  Education,  46(6),  50-55. 

Farnham-Diggory,  S.  (1994).  Paradigms  of  knowledge  and  instruction.  Review  of  Educational  Research.  M(3),  463-477. 

Gagnd,  R.  M.,  Briggs,  L.  J.,  & Wager,  W.  W.  (1992).  Principles  of  instructional  design  (4  ed.).  Orlando:  Harcourt, 
Brace,  Jovanovitch. 

Hamilton,  G.  H.  (1972).  19th  and  20th  century  art:  Paintine.  sculptur_e_architecture,  Englewood  Cliffs,  NJ:  Prentice-Hall. 

Hooper,  S.,  & Rieber,  L.  P.  (in  press).  Teaching  with  technology.  In  A.  Ornstein  (Ed.),  Teaching:  Theory  and  practice 
Boston:  MA:  Allyn  and  Bacon. 

Jonassen,  D.  (1991a).  Evaluating  constructivist  learning.  Educational-Technology  Researchjind  Development.  21(9),  28- 
33. 

Jonassen,  D.  (199io).  Objectivism  versus  constructivism:  Do  we  need  a new  philosophical  paradigm?.  Educational 
Technology  Research  and  Development,  22(3),  5-14. 


Perkins,  D.  N.  (1994).  The  intelligent  eve:  Learning  to  think  by  looking  at  art.  Santa  Monica:  The  J.  Paul  Getty  Trust. 
Richey,  R.  (1986).  The  theoretical  and  conceptual  bases  of  instructional  design.  London:  Kogan  Page  Ltd. 

Rieber,  L.  P.,  & Welliver,  P.  W.  (1989).  Infusing  educational  technology  into  mainstream  educational  computing. 
International  Journal  of  Instructional  Media.  16(1).  21-32. 

Salomon,  G.  (1991).  Transcending  the  qualitative  debate:  The  analytic  and  systemic  approaches  to  educational  research. 
Educational  Researcher.  2Q(6),  10-18. 

Tennyson,  R.  D.,  & Rasch,  M.  (1988).  Linking  cognitive  theory  to  instructional  prescriptions.  Instructional  Science. 
12.  369-385. 

Wager,  W„  Lebow,  D.,  Driscoll,  M„  & Supinski,  S.  (1995).  Designing  constructive  environments:  Principles. 

technology,  assessment  and  transferability.  Paper  presented  at  the  national  convention  of  the  Association  for 
Educational  Communications  and  Technology,  Annaheim,  CA. 

Winn,  W.  (1993).  Instructional  design  and  situated  learning:  Paradox  or  partnership?.  Educational  Technology.  22(3), 

16-21. 


i r1  .i  • 


632 


Title: 


Six  stages  for  learning  to  use  technology 


Author: 

Anne  L.  Russell 

Queensland  University  of  Technology 


ABSTRACT 


Learning  to  use  technology  for  adults  can  be  traumatic.  This  paper  describes  six  stages  adults  may  go  through  as.they  learn 
to  use  technology  to  communicate  electronically. 

An  action  research  model  has  been  used  to  develop  a strategy  for  adult  learners  to  understand  the  stages  they  will  go  through 
as  they  leant  to  use  electronic  mail  as  a communication  tool.  There  are  two  cycles  in  the  study  reported  in  this  paper.  First, 
is  the  identification  and  description  of  six  stages  learners  go  through  asthe  technology  moves  from  being  intrusive  to 
becoming  invisible.  And  secondly,  is  the  application  of  metacognitive  understanding  of  these  stages  by  new  naive  adults 
learners  as  they  use  the  technology. 

When  new  technology  learners  apply  the  knowledge  of  these  six  stages  to  their  learning,  they  appreciate  they  are  not  unique 
and  inadequate  in  their  ability  to  come  to  terms  with  the  technology.  In  addition, teachers  of  technology  who  have  knowledge 
of  the  six  stages  can  identify  the  key  times  when  they  need  toprovide  intensive  support  with  the  knowledge  thatthe  learner 
will  require  less  suppcU  during  the  later  stages. 

INTRODUCTION 

Access  to  technology  can  "allow  people  separated  by  distance  and  time  to  occupy  a shared  synthetic  world,  where  they  can 
collaborate  to  evolved  common  virtual  experiences"  (Dede  in  O'Neil,  1995,  p.l  1).  In  order  for  educators  to  keep  up  with,  and 
to  be  actively  involved  in,  national  and  international  trends,  it  is  an  advantage  to  have  the  ability  to  communicate  with 
colleagues  across  the  globe.  However,  in  the  information  age  where  the  Internet  may  be  a way  of  life  for  the  young  child, 
many  adults  find  learning  to  use  computer  technology  is  very  daunting.  Adults  who  are  not  familiar  with  technology  can  feel 
left  behind.  Special  attention  needs  to  be  given  to  ensure  they  learn  without  loosing  self-esteem  andwithout  dropping  out  all 
together.  Simonson,  Maurer,  Montag-Rorardi,  & Whitaker,  (1987)  identified  'that  a positive,  anxiety  free  attitude  toward 
computing  was  a necessary  prerequisite  of  computer  literacy'  (p.233). 

Researchers  (Rosen,  Sears, & Weil,  1993;  Mclnerney,  Mclnerney,  & Sinclair,  (1994)  studied  university  students'  levels  of 
computer  anxiety  before  and  after  taking  computer  courses.  The  type  of  computer  course  has  been  an  important  factor  in 
reducing  anxiety  (Leso  & Peck,  1992).  However,  reducing  anxiety  through  learning  computer  applications  using  a relevant 
activity  combined  with  an  understanding  of  stages  a learner  typically  goes  through  during  the  learning  process  has  not  been 
considered.  Learning  in  this  context  represents  the  new  understanding  and  skills  which  occur  when  technology  hardware  and 
the  software  processes  move  from  being  intrusive  and  frustrating  to  being  invisible.  This  allows  an  individual  to  develop 
confidence  and  consider  creative  uses  of  the  technology. 

The  study  reported  in  this  paper  is  qualitative  and  based  on  personal  e-mail  diaries  written  by  adult  learners.  It  focuses  on  the 
identification  of  learning  stages  in  these  students  self-reports  as  they  use  e-mail  for  first  time.  The  second  phase  of  the  study 
involved  introducing  the  identified  stages  to  a new  group  of  students  also  learning  to  use  e-mail. 

Computer  Anxiety 

In  the  information  age  adults  believe  they  should  be  computer  literate.  Many  feel  inadequate  about  their  abilities  to  develop 
computer  skills  and  actively  avoid  using  computers.  Evans-Andris  (1995)  observed  elementary  teachers  and  found: 

Approximately,  62%  of  the  teachers  tended  to  engage  primarily  in  distancing  routines,  limiting  their  involvement  with 
computers,  whereas  approximately  38%  of  the  teachers  engaged  primarily  in  embracing  routines,  increasing  their 
opportunities  to  use  the  equipment,  (p.  20) 

Kolehmainen  (1992)  suggested  computer  anxiety  will  have  negative  effects  when  learning  new  technology  because  of  a 
resistance  to  change.  If  this  is  so,  it  may  be  important  to  identify  for  learners  a relevant  purpose  for  learning  which  reflects 
and  reinforces  their  current  values.  Involving  students  in  a relevant  task  where  the  technological  processes  are  merely  a means 
to  an  end,  may  lead  to  overcoming  computer  anxiety  earlier. 
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Rosen  et  al.  (1993)  and  Mclnerney  et  al.  (1994)  found  first-year  university  students  in  the  1990s  to  be  computer  anxious  as 
they  commence  their  compulsory  computer  studies.  Both  these  studies  reported  many  of  the  students  still  remained  anxious  at 
the  completion  of  their  computer  course. 

A computer  literacy  and  computer  anxiety  test  has  been  devised  by  Simonson  et  al.  (1987)  and  has  been  used  to  measure  the 
level  of  computer  anxiety  in  university  students  before  and  after  completing  a computer  course. 

The  type  of  computer  course  can  have  some  influence  on  the  level  of  anxiety  of  graduating  students.  Leso  and  Peck  (1992) 
found  students  taking  a software  tool  course  (included  word  processing,  spreadsheets,  etc.)  were  more  likely  to  come  out  with 
reduced  anxiety  than  those  students  doing  a programming  course.  However,  the  study  did  not  explore  the  reason  for  this 
outcome.  In  fact,  many  students  did  not  reduce  their  level  of  anxiety  by  the  end  of  either  course. 

"Increased  computer  experience  does  not  necessarily  alleviate  anxiety"  is  also  reported  by  Mclnerney  et  al.  (1994.  p.  47). 

These  authors  suggested  researchers  focus  "on  building  confidence  and  a sense  of  personal  control  in  a non-threatening 
learning  environment,  individualized  if  necessary"  (1994.  p.47).  It  appears  the  processes  of  learning  to  use  computers  can 
cause  a decrease  in  self-esteem. 

Becoming  Computer  Literate 

Being  computer  literate  implies  being  able  to  use  a computer  and  to  apply  current  understanding  to  new  situations.  In 
relation  to  this  paper,  computer  literacy  refers  to  common  hardware  and  software  'tools*  programs  and,  in  particular,  using  e- 
mail.  In  early  computer  experiences  the  hardware  and  software  are  intrusive.  As  the  learner  understands  how  to  manipulate 
the  computer,  more  control  over  the  technology  enables  the  focus  to  shift  to  tasks  to  be  completed. 

Valdez  (1989)  suggested  that  "technology  should  be  the  means  by  which  teachers  and  students  are  able  to  feel  more 
empowered  and  in  control  of  their  lives"  (p.  38).  He  identified  awareness,  adoption  and  refinement/adaptation  as  "three  stages 
that  most  people  experience  when  learning  to  use  technology"  (p.37).  The  present  study  reports  finding  six  stages  which 
learners  go  through  as  the  hardware  and  software  processes  move  from  intrusive  to  invisible. 

Learning  to  use  technology  in  this  paper  is  presented  from  a constructivist  perspective  where  learning  occurs  through 
immersing  the  learner  in  a real  and  relevant  experience  with  collaborative  faculty  and  peer  support.  The  learner  brings 
metacognitive  understanding  to  the  learning  situation  to  help  overcome  some  of  the  frustrations  of  learning  and  bring 
understanding  to  the  task. 

The  learner  constructs  personal  meaning  through  immersion  in  the  situation  and  develops  strategies  for  building  an 
understanding  of  the  practicalities  of  the  technological  processes.  Apart  from  the  physical  presence  of  the  technology  there  are 
other  sources  for  help  which  range  from  printed  manuals  to  faculty  and  peer  support.  The  learner  develops  an  understanding  of 
how  the  technology  operates  and  identifies  the  steps  required  to  make  it  work  in  order  to  achieve  success  in  the  processes. 

The  understanding  will  be  such  that  the  learner  can  accommodate  flexibly  when  varied  processes  are  required. 

In  the  study  reported  here  the  students  were  given  operation  manuals,  personal  instruction,  provided  with  faculty  and  peer 
support,  but  were  encouraged  to  create  their  own  mental  schema  and  processes  to  understand  the  technology.  This  is  earned 
out  through  the  immersion  in  a real  and  relevant  task  which  involves  children  from  schools  across  Australia. 

THE  STUDY 

Initially  thirty  teachers  studying  in  a post-graduate  university  course  were  involved  in  a compulsory  assignment  which 
required  them  to  learn  to  use  e-mail.  The  vehicle  for  their  learning  was  to  take  the  role  of  a character  from  a children's  fiction 
book  and  respond  to  letters  from  school  children.  This  Characters-on  Line  Project  during  a three  week  period  provided  a 
service  to  Australian  schools  (Russell,  in  press).  All  communications  took  place  using  e-mail.  In  addition  to  e-mailing  to 
the  children,  each  student  was  required  to  send  three  e-mail  message  to  their  professor  explaining  how  they  were  learning  to 
use  the  technology.  These  messages,  or  metacognitive  learning  diaries,  were  sent  at  intervals  throughout  the  learning 
experience. 
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Students  came  into  the  course  with  varying  technological  knowledge.  While  some  had  actively  avoided  using  computers, 
others  had  practical  experience  in  word-processing  and  school  library  automation  activities.  No  student  had  previously 
accessed  electronic  mail. 

Initially  all  students  were  on-campus  attending  university  full-time.  Over  the  following  years  the  nature  of  the  course 
changed  and  the  majority  of  the  post-graduate  students  became  part-time  external  or  open  learning  students,  many  of  whom 
live  in  remote  locations.  They  accessed  e-mail  and  completed  the  project  assignment  with  a borrowed  modem  from  their 
home  or  at  their  local  school. 

Since  the  initial  thirty  teachers  completed  their  e-mail  assignment,  some  400  students  have  repeated  the  process  and  e-mailed 
metacognitive  reflections  on  their  learning  experiences.  The  e-mailed  metacognitive  reflections  from  students  formed  a pool 
of  data  which  was  analysed  using  grounded  theory  (Lincoln  & Guba,  1985).  Categories  of  metacognitive  reflections  on  the 
experience  of  learning  technological  skills  related  to  using  electronic  mail  were  identified.  Six  categories  emerged  from  the 
diary  reports  and  these  formed  stages  which  learners  typically  go  through  as  they  learn  to  use  technology: 

Stage  1:  Awareness 

Stage  2:  Learning  the  process 

Stage  3:  Understanding  and  application  of  the  process 

Stage  4:  Familiarity  and  confidence 

Stage  5:  Adaptation  to  other  contexts 

Stage  6:  Creative  application  to  new  contexts. 

In  the  second  stage  of  the  study  these  six  stages  were  presented  to  new  students  as  they  commenced  their  email  experience  and 
learned  to  use  the  technology.  These  students  were  asked  to  reflect  on  their  personal  learning  in  relation  to  the  identified  six 
stages.  This  second  aspect  of  the  action  research  cycle  validates  the  six  stages  as  students  reported  similar  feelings  of 
frustration  and  success.  This  aspect  will  be  explored  after  describing  in  more  detail  the  six  stages  for  learning  to  use 
technology. 

PHASE  ONE:  STAGES  FOR  LEARNING  TO  USE  TECHNOLOGY 

The  six  stages  learners  usually  pass  through  as  e-mail  technology  moves  from  the  intrusive  to  the  invisible  are  described  and 
supported  by  illustrations  from  metacognitive  reports  of  the  students. 

Stage  1:  Awareness 

Here  students  are  aware  that  electronic  mail  exists.  Many  students  have  actively  avoided  using  computers  and  no  student 
reported  using  e-mail  prior  to  this  compulsory  activity.  A sense  of  nervousness  and  fear  was  in  the  minds  of  some  students 
while  others  were  looking  forward  to  having  the  opportunity  to  do  something  they  had  heard  about. 

I had  no  idea  what  was  actually  going  to  occur.  I had  heard  of  electronic  mail,  but  had  never  seen  it  in  operation.  Because  of 
this,  I felt  rather  bewildered  and  to  a certain  extent  concerned. 

The  very  thought  of  such  technology  - computers,  satellites,  etc.,  was  extremely  daunting.  I was  also  excited  by  the  idea  that 
I would  have  to  be  involved  and  get  to  know  all  about  it. 

Stage  2:  Learning  the  Process 

Reports  of  time-consuming  assimilation  of  new  information  as  new  skills  are  mastered  epitomizes  this  stage.  Instructions 
are  often  misleading  for  the  novice  who  does  not  understand  the  processes.  Frustrations  with  complex  technology  renders  the 
equipment  intrusive  in  the  eyes  of  the  user.  The  technology  can  overpower  and  intimidate  the  learner. 

Some  students  without  extensive  computer  experiences,  are  afraid  of  damaging  equipment.  Others,  in  hindsight  enjoy  this 
challenge  and  persevere  to  ably  move  beyond  this  stage.  Most  students  find  working  within  a group  is  extremely  valuable  for 


providing  moral  support.  Apart  from  peer  support,  this  stage  requires  extensive  technical  and  positive  encouragement  from 
the  teacher. 

We  experienced  a variety  of  emotions:  extreme  frustration  often  when  there  appeared  to  be  no  logical  explanation  for  things 
going  wrong;  annoyance  about  the  amount  of  time  wasted. 

The  handbooks  were  meaningless,  until  I had  some  'hands-on’  experience. 

Those  experiencing  problems  only  had  to  ask  and  someone  was  sure  to  be  there  with  a suggestion  or  hint. 

Stage  3:  Understanding  and  Application  of  the  Process 

As  the  learner  begins  to  understand  the  logic  behind  the  technological  processes,  the  need  to  cling  to  step  by-step  instructions 
can  be  relaxed.  Where  early  instructions  had  no  meaning  and  were  therefore  forgotten,  now  the  learners  have  'hands-on' 
experience  and  they  can  accommodate  new  instructions  within  their  basic  understanding.  The  relevance  of  the  task  gives 
added  incentive  with  a purpose  to  continue  learning  the  processes.  A sense  of  community  is  felt  with  the  presence  of  peers 
who  are  also  learning  the  processes  and  providing  moral  support. 

After  plodding  through  a couple  of  weeks,  our  knowledge  of  [e-mail]  had  increased  and  so,  this  time,  some  of  the  things 
began  to  makes  sense,  and  suddenly  we  could  see  where  we  had  gone  wrong.  The  next  attempt  brought  success! 

Stage  4:  Familiarity  and  Confidence 

With  confidence  the  technological  processes  are  applied  to  the  task  at  hand  as  the  learner  can  visualize  the  processes  and 
anticipate  logical  outcomes  for  certain  software  commands  or  apparent  inconsistencies.  The  technology  is  starting  to  become 
transparent  when  problems  become  'hiccups’  rather  than  major  distracters. 

Sometimes  there  is  a reversion  to  earlier  stages,  but  confidence  is  more  quickly  regained.  Self-esteem  is  increased  as  the 
learner  is  able  to  solve  the  problems  through  familiarity  or  logical  analysis  of  the  situation.  Working  things  out  alone  give  a 
sense  of  knowing,  understanding  and  confidence. 

By  the  beginning  of  the  second  week  I could  successfully  work  on  [e-mail]  without  the  use  of  the  much  valued  manual  - 
something  I was  quite  proud  of. 

Having  overcome  the  problems  of  operating  the  system  efficiently,  I could  then  concentrate  on  the  process  of  the  task: 
assuming  the  role  of  the  character  of  Wilbur.  This  was  one  of  the  highlights.  Being  able  to  communicate  with  children  and 
sharing  with  them  their  thoughts  and  feelings  on  the  book  " Charlotte  s Web  . 

Stage  5:  Adaptation  to  Other  Contexts 

Now  the  technology  and  software  are  invisible  and  the  learner  can  see  the  potential  for  use  of  e-mail  in  other  curriculum 
situations.  New  understandings  and  experiences  are  transferred  to  other  contexts.  For  example,  one  student  reports  a new 
sense  of  confidence  when  she  assists  school  children  as  they  learn  a word  processing  package  she  has  not  previously  seen. 

Now  that  I know  how  to  operate  the  system,  I hope  that  I will  have  the  opportunity  to  work  on  future  email]  projects.  As 
well,  this  time  has  also  made  it  evident  to  me  that  I will  need  to  become  more  familiar  with  computers  and  associated  pieces 
of  equipment,  if  I am  going  to  incorporate  them  successfully  into  my  resource  centre. 


Stage  6:  Creative  Application  to  New  Contexts. 

This  stage  is  reached  by  students  for  whom  the  technological  processes  become  invisible  and  who  naturally  use  electronic 
mail  to  extend  their  educational  environment  when  this  is  appropriate. 
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Several  years  after  this  initial  contact  with  e-mail,  some  students  have  reported  that  the  experience  led  themto  become  the 
computer  coordinator  in  their  school.  Other  students  have  sent  future  assignments  via  e-mail  when  this  was  not  an 
expectation.  Recognizing  implications  and  possibilities  for  other  uses  of  e-mail  was  also  evident. 

Electronic  mail  could  provide  deaf  children  with  the  facility  to  participate  in  the  visual  form  of  natural  language  with  their 
peers.  It  would  also  widen  the  audience  for  outback  students. 

PHASE  TWO:  METACOGNITIVE  APPLICATION  OF  THE  STAGES 

Once  the  six  stages  most  students  go  through  as  they  learn  to  use  the  technology  were  identified,  new  students  were 
introduced  to  the  stages  at  the  beginning  of  their  on-line  assignment.  The  diary  reports  they  send  reflected  their  personal 
learning  in  relation  to  the  six  stages.  The  intention  was  to  provide  the  opportunity  for  the  students  to  articulate  their  personal 
learning  strategies  and  also  to  validate  the  six  stages  as  being  applicable  for  learners  of  new  technology. 

My  reflection  on  their  metacognitive  insights  helped  me  to  identify  how  better  to  assist  these  and  future  students  as  they  learn 
new  technological  processes. 

The  following  discussion  epitomizes  e-mail  diaries  of  sixty  students  doing  the  Characters-on-Line  Project  in  1995.  It  should 
be  acknowledged  that  some  students  may  not  like  to  admit  their  feelings  of  inadequacy.  However  there  are  sufficient  positive 
reflections  to  indicate  students’  computer  anxiety  is  overcome  at  the  completion  of  the  task  and  confirmation  of  the  six  stages 
for  learning  to  use  new  technology. 

A case  study  of  Pat  will  be  presented  here  with  her  references  to  the  six  stages  as  she  was  learning  to  use  e-mail.  Her  insights 
were  typical  of  the  other  students.  The  following  message  was  sent  to  me  six  weeks  after  afive  hour  introduction  and  hands 
on  training  session  during  an  on-campus  session  at  the  university.  Pat  is  a full-time  teacher-librarian  (media  specialist)  living 
in  a country  town  too  far  to  return  to  the  university  for  assistance  in  using  e-mail.  She  was  a member  of  the  team  who  were 
took  on  the  character  of  eight  year  old  Jason  from  a picture  book  called  Our  Excursion  by  Kate  Walker  & David 
Cox  (Omnibus  Books).  As  ’’Jason”,  Pat  replied  to  letters  from  school  children  across  Australia. 

Stage  1:  Awareness 

All  my  initial  fears,  phobias  and  woes  seem  long  forgotten  and  I’m  a bit  sad  to  be  saying  good-bye  to  Jason  and  the  [e-mail] 
kids.  I jotted  down  a few  notes  regarding  the  six  stages  you  propose.  In  Stage  1 I certainly  had  mixed  feelings  about  what 
was  going  to  occur.  The  best  way  to  describe  it  I guess  is  that  it  was  a strange  mix  of  excitement  and  terror  - do  you  know 
that  feeling? 

As  I read  the  comments  from  students,  I am  conscious  of  their  anxiety  at  the  beginning  of  the  project.  Many  have  avoided 
using  computers  and  others  are  afraid  of  the  technology  and  the  damage  they  will  do  to  the  hardware.  These  students  need  to 
have  useful  instructions,  equipment  that  works  andaccess  to  someone  who  knows  the  particulars  of  their  computer  setup.  At 
the  same  time  there  are  students  who  are  confident  and  keen  to  extend  their  experiences  with  the  technology. 


Stage  2:  Learning  the  process 

I felt  fairly  confident  when  I left  uni  (how  those  other  students  managed  with  a phone  workshop  I’ll  never  know  - 1 guess  I’m 
a hands  on  learner),  but  when  I got  home  I had  trouble  setting  up  the  system  and  our  shared  phone  line  crashed  for  a few  day. 
It  threw  me  and  I lost  confidence.  In  Stage  2 I was  pretty  much  frustrated  and  impatient.  I felt  uncertain  and  suddenly  all  the 
instructions  that  had  seemed  clear  at  uni  didn’t  seem  to  make  sense. 

Stage  Two  requires  direct  non-threatening  assistance  from  someone  who  has  knowledge  of  the  student’s  individual  computer 
setup.  Preferably  this  is  someone  who  is  readily  available  on  site  or  via  telephone.  It  is  this  stage  where  I receive  many 
phone  calls  for  assistance.  Observation  of  two  naive  learners  sitting  together  at  one  keyboard  is  fascinating.  The  person  at 
the  keyboard  seems  to  have  an  empty  mind  and  the  observer  knows  all  the  answers  and  steps  in  the  process.  They  then 
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exchange  positions  and  the  empty  minded  person  has  all  the  answers  as  the  former  expert  sits  at  the  keyboard  with  an  empty 
mind! 

This  stage  requires  confidence  building.  It  is  essential  students  do  not  feel  or  be  made  to  feel  stupid.  A phrase  I use  often 
during  this  stage  is,  "The  more  mistakes  you  make  the  more  you  will  learn."  Permission  and  even  encouragement  to  make 
mistakes  is  essential  and  acknowledges  that  others  are  equally  unaware. 

Stage  3:  Understanding  and  Application  of  the  Process 

Stage  3 was  reassuring  because  gradually  things  started  to  fall  into  place.  Although  I felt  that  I didn't  know  where  my 
information  was  at  any  one  time  ( cyberspace????  what  is  that  ???)  I gradually  was  coming  to  grips  with  the  [e-mail]  concept. 

I learned  the  hard  way  to  be  accurate. 

Less  help  is  required  from  experts  at  this  stage.  Often  students  turn  to  each  other  via  e-mail  or  telephone  for  assistance.  They 
build  confidence  as  they  help  each  other  overcome  problems.  My  presence  is  seldom  sought. 

Stage  4:  Familiarity  and  Confidence 

Stage  4 was  great.  I was  up  and  rolling  and  really  having  fun.  I was  in  character  with  Jason  and  starting  to  develop  a 
fondness  for  him.  I enjoyed  getting  the  kid's  letters  and  even  wrote  to  a couple  and  asked  them  to  write  back  to  me  and  tell 
me  their  dog's  name  etc.  One  letter  was  quite  sad,  about  a recently  deceased  pet  dog,  and  I felt  Jason  could  identify  with  how 
he  would  feel  if  his  dog  died  and  maybe  in  some  small  way  help  the  grieving  of  that  child  -Who  knows?  I enjoyed  the 
freedom  of  the  character  and  talking  in  an  eight  year-old  way  to  the  kids  - don't  show  this  to  any  psychiatry  colleagues! 

Even  the  way  this  is  written  reflects  the  freedom  experienced  when  the  technology  begins  to  turn  invisible  and  the  task 
becomes  the  focus  of  attention.  Letters  I receive  at  this  stage  are  longer  and  the  students  talk  about  their  family  and  leisure 
time  pursuits  in  addition  to  their  involvement  in  the  task. 

Stage  5:  Adaptation  to  other  contexts 

Stage  5 - 1 started  to  think  about  how  1 could  use  [e-mail]  in  my  job  -perhaps  to  contact  authors  etc.  I can  see  that  it  will  be 
possible  to  use  [e-mail]  to  communicate  with  other  teacher-librarians  etc.  Is  there  a directory  of  users  available  ? 

Here  Pat  is  moving  beyond  the  immediate  task  and  exploring  the  potential  for  relevant  use  of  e-mail  in  other  aspects  of  her 
job  as  a media  specialist  in  a school.  By  now  the  technology  is  invisible  and  only  a vehicle  through  which  the  wider  world 
can  be  reached  for  personal  professional  development  and  for  the  benefit  of  student  learning. 

Stage  6:  Creative  Application  to  New  Contexts 

(Stage6).  L 

Thanks  for  the  opportunity  to  [e-mail]  I have  enjoyed  it  and  it  has  given  me  a common  experience  to  share  with  my  student 

colleagues  in  the  oft’  lonely  land  of  external  study.  Cheers,  Pat. 

Being  a member  of  an  e-mail  team  has  had  a side  benefit  where  colleagues  have  been  brought  together  and  shared  problems 
and  so  come  to  know  each  other.  In  the  future  they  may  continue  to  be  in  touch  through  e-mail  and  so  a network  of 
professional  communication  has  commenced. 

Pat  can  not  reflect  on  Stage  Six  yet  as,  by  its  very  nature,  it  will  not  be  evident  until  some  time  after  this  project  is  ended. 
Then  Pat  may  find  she  wants  to  achieve  a particular  vision  and  e-mail  will  be  the  most  obvious  way  to  achieve  it.  During 
the  semester  I have  received  requests  for  extensions  and  clarification  related  to  assignments  for  other  courses  from  students 
who  were  originally  introduced  to  e-mail  as  a Character-on-Linc. 
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While  many  students  reported  concluding  the  project  with  positive  computer  learning  and  less  anxiety,  no  formal  measure  of 
computer  anxiety  either  pre  or  post  project  was  administered.  Some  students  chose  not  t'  vri'ce  about  their  learning  in 
relation  to  the  six  stages.  No  close  record  was  kept  of  students  who  claimed  to  be  anxious  at  the  beginning  and  their  outcome 
comments  at  the  end.  As  with  many  e-mail  messages  the  request  for  information  is  often  not  responded  to.  It  could  be 
possible  that  students  not  reporting  their  computer  experiences  were  either  still  anxious  or  the  technology  was  so  invisible 
they  did  not  consider  reporting  to  be  of  any  consequence. 

CONCLUSION 

The  six  stages  of  learning  to  use  a new  technology  has  been  validated  through  this  study.  The  students  seem  to  jump  the 
hurdle  of  the  computer  anxiety  in  order  to  put  their  minds  to  the  authentic  task  of  being  a character  in  a book  and  writing 
letters  to  real  students  in  schools  across  Australia. 

This  empirical  study  is  as  applicable  for  teachers  of  computer  anxious  learners  as  it  is  for  the  learners  themselves.  Four 
recommendations  are  suggested  for  instructional  designers  of  courses  for  naive  computer  users: 

1.  Use  a novel  relevant  real-life  task  to  encourage  the  student  to  jump  the  hurdle  of  the  processes  in  order  to  become 
involved  in  the  task.  Leso  and  Peck  (1992)  found  students  learning  word-processing  and  other  tool  skills  likely  to  be  less 
computer  anxious  at  the  end  of  the  course.  However,  they  still  found  a third  of  these  students  measured  high  on  the  computer 
anxiety  scale.  Perhaps  a pre  and  post  computer  anxiety  measurement  where  the  course  included  a novel  and  relevant  or 
authentic  task  as  an  outcome  would  find  a lower  number  of  computer  anxious  students  at  the  completion  of  the  task. 

2.  Introduce  the  six  stages  for  learning  a new  technology  to  students  before  they  commence  a computer  course. 
Frequently  students  report  feeling  inadequate  and  stupid  and  expect  to  be  experts  with  their  first  computer  experiences. 

Through  knowing  the  stages  they  may  relate  to  the  frustrations  of  previous  students  and  remove  some  of  the  pressure  to  be 
perfect  at  the  first  attempt.  Knowledge  that  they  will  eventually  succeed  as  other  students  have  done  can  remove  some  of  the 
stress. 

3 . Provide  a non-threatening  environment  with  extensive  technical  and  moral  support  during  Stages  Two  and  Three 
when  self-esteem  can  be  low  and  problems  are  not  easily  solved  by  a naive  learner.  Mclnemey  et  al.  (1994)  suggested  students 
develop  confidence  if  the  learning  en  vironment  is  non-threatening.  At  this  time  there  is  a need  for  mediation  and  coaching.  It 
is  preferable  for  a non -threatening  expert  to  be  readily  available  on  site,  but,  if  this  is  not  possible,  telephone  contact  with  the 
teacher  or  another  expert  is  appropriate.  However,  when  students  are  unable  to  contact  an  expert  they  find  solace  and 
problems  are  solved  through  talking  or  e-mailing  with  a student  colleague  who  is  learning  the  same  processes.  It  seems  a 
learning  community  can  be  developed  which  builds  confidence  and  understanding  in  all  members  of  that  group.  This  idea 
parallels  Dede's  suggestion  (in  O’Neil,  1995)  of  students  sharing  a ’synthetic  world’. 

4.  Encourage  students  to  articulate  their  metacognitive  experiences  in  relation  to  the  six  Stages.  When  students 
verbalize  their  learning,  they  can  identify  where  they  have  come  from,  where  they  are  now  and  the  vision  of  success.  The  act 
of  interacting  with  the  six  stages  in  order  to  clarify  their  own  learning  situation  objectifies  their  learning  and  may  remove 
negative  subjective  thoughts. 

In  vhe  future  the  technological  processes  may  not  be  electronic  mail,  but  another  technological  innovation  which  needs 
learning  through  coaching  and  support  with  frustiating  and  time  consuming  focus  on  the  processes  before  the  technology 
invisibly  becomes  incorporated  within  an  environment.  Only  then  can  creative  and  worthwhile  uses  be  applied  in  a variety  of 
contexts. 
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Abstract 


The  Internet  promises  dramatic  changes  in  the  way  we  learn  and  teach,  the  way  we  interact  as  a society.  The  changes  of  a 
coming  epoch  are  already  taking  shape.  In  higher  education,  most  students  have  access  to  Internet  resources.  But  many 
who  have  access  have  shown  reluctance  to  make  use  of  those  resources.  An  attempt  to  understand  this  reluctance  has  led 
to  an  analysis  using  James  Gibson’s  model  of  affordances.  This  paper  contrasts  the  Internet  with  other  fundamental 
educational  infrastructures  throughout  history.  It  offers  an  analysis  of  literacy,  printing  and  electronic  publishing  in  terms 
of  human  affordances  along  with  the  constraints  associated  with  each  medium. 

Affordances  and  Constraints  of  the  Internet 

The  history  of  Education  cannot  be  told  apart  from  its  technology.  From  orality  to  literacy,  from  manuscripts  to  the 
printed  page,  from  text  to  hypertext,  the  prevailing  technologies  supporting  education  have  defined  its  very  nature. 
Stopping  short  of  technological  determinism,  we  acknowledge  the  chicken-and-egg  relationship  between  technology, 
education,  and  society.  The  possibilities  which  education  brings  to  society  depend  upon  the  affordances  of  the 
undergifding  technology. 

We  use  the  term  affordance  to  describe  a potential  for  action,  the  perceived  capacity  of  an  object  to  enable  the  assertive 
will  of  the  actor.  The  term  was  coined  by  psychologist  James  Gibson  (1977)  to  describe  the  action  possibilities  posed  by 
objects  in  the  real  world.  There  are  many  objects  in  our  environment.  Some  we  ignore,  some  we  adapt  to,  and  some  we 
appropriate  for  our  assertive  will.  It  is  the  objects  in  this  last  category  which  fall  under  the  the  definition  of  affordances. 
Certain  objects  afford  opportunities  for  action.  An  affordance  is  a value-rich  ecological  object  that  is  understood  by  direct 
perception.  Perception  informs  the  individual  of  affordances.  Action  transforms  affordances  into  effectivities  which  extend 
human  capabilities  (Allen  and  Otto,  1995).  Our  own  bodies  are  affordances.  The  eyes  afford  perception,  the  ears  listening, 
the  hands  manipulation,  the  tongue  and  vocal  cords  afford  utterances  (Jonassen,  Campbell  and  Davidson,  1994).  Natural 
affordances  emerge  into  effectivities  through  use  in  conscious  activity.  The  hand  of  an  infant,  though  attached,  is  a 
separate  object.  The  infant  is  amused  by  it,  studies  it,  tastes  it,  touches  other  things  with  it.  Soon  the  infant  learns  to  use 
the  hand  to  manipulate  other  objects.  In  the  process,  the  hand  gradually  transforms  its  object-ness  to  subject-ness.  The 
child  becomes  less  conscious  of  the  hand  as  she  uses  it  as  an  extension  of  her  own  intentioned  will.  The  affordance 
becomes  an  effectivitv. 

Technology  and  media  are  affordances  to  the  extent  that  they  promise  extended  human  capabilities  of  seeing,  hearing,  and 
uttering.  Tools  are  affordances  to  the  extent  they  offer  extended  human  capabilities  for  manipulating  things  in  the 
environment.  (Rasmussen,  et.  al.,  1994).  Through  use,  skill  is  acquired  and  the  object  becomes  an  extension  of  ourselves 
(MeCluhan,  1964).  These  artifacts  are  transformed  from  affordances  to  effectivities. 

Literacy 

Five  thousand  years  ago  the  affordances  of  orality  and  rhetoric  were  extended  by  the  technology  of  written  text.  Writing 
enabled  the  precision  of  recorded  memory.  It  afforded  cognitive  communication  across  time  and  distance.  Literacy  gave 
birth  to  History.  The  written  word  introduced  constraints  of  grammar  and  syntax,  but  out  of  these  constraints  emerged 
stability  and  permanence  of  language.  It  was  a means  of  representing  and  archiving  human  knowledge  for  later  retrieval 
by  one's  self  or  by  another. 

We  look  at  antiquity  through  the  eyes  of  those  who  could  read  and  write,  shading,  perhaps,  our  view  of  the  importance  of 
literacy  in  shaping  history.  When  we  think  of  antiquity,  Athens  invariably  comes  to  mind.  Athens  was  among  the  few 
ancient  cultures  that  valued  literacy.  Athens  provided  a system  of  public  education  which  included  reading  and  writing,  an 
affordance  we  take  for  granted  today.  But,  just  as  it  is  today,  not  all  Athenians  chose  to  appropriate  skills  of  literacy. 
Athens  produced  history's  most  famous  illiterate,  Socrates,  who  opposed  the  technology  of  writing  on  the  grounds  that  it 
would  corrupt  the  human  mind  and  destroy  the  memory  of  mankind  (Plato,  360  BCE).  We  suggest,  from  the  standpoint 
of  affordance,  that  Socrates  perceived  no  need  to  write.  After  all,  he  had  his  graduate  students,  Xenophon  and  Plato,  to 
record  his  words  for  him(l).  But  more  fundamentally  the  assertive  will  of  Socrates  found  its  full  potential  within  the 
affordance  of  oral  discourse  and  dialog.  Writing,  to  Socrates,  was  an  aimless  diversion.  Indeed,  for  most  of  written 
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history,  writing  was  not  an  affordance  for  the  common  person.  For  centuries,  it  was  the  exclusive  realm  of  the  power 
elite. 

The  manuscript  was  an  instrument  of  power.  The  written  word  is  a permanent  record,  an  objective  archive,  a sense  of 
authority.  Manuscripts  were  guarded,  protected  and  generally  inaccessible  (Mukerji  and  Jones,  1995).  For  thousands  of 
years,  written  knowledge  was  the  affordance  of  the  literate  few:  the  priests  and  princes  who  controlled  access  to 
manuscripts  and  who  had  the  skills  to  make  use  of  the  technology.  Those  who  controlled  knowledge  controlled  the 
collective  conscience,  the  laws,  the  social  structures,  and  the  tax  records.  Literacy  was  the  means  of  social  control. 

Medieval  to  Modern 

In  the  period  approaching  the  late  Middle  Ages,  the  power  of  literacy  began  to  spread  to  the  merchant  class.  Universities 
emerged  in  the  thirteenth  century,  before  the  age  of  Gutenberg.  Manuscripts  were  extremely  costly,  reproduction  of  a 
single  copy  required  weeks  of  labor  by  an  established  member  of  the  literati  (Eisenstein,  1983).  Often,  students  copied 
their  own  texts,  transcribing  oral  recitations  by  a lecturer/reader(2).  Learning  to  read  and  write  was  the  privilege  of  an  elite 
few.  For  those  without  means,  a literate  education  was  characterized  by  extreme  effort  supported  only  by  primitive 
techniques  and  rudimentary  tools(3).  Entry  into  the  literate  community  was  not  possible  without  great  personal  sacrifice 
(Cantor,  1969).  Manuscripts,  the  objects  that  enabled  this  intention,  were  affordances  for  acquiring  literacy. 

The  rise  of  the  universities  in  Europe  and  the  neo-classicism  associated  with  the  return  of  the  Crusaders  in  the  thirteenth 
century  drove  unprecedented  demands  on  manuscript  production.  In  a previous  era,  this  renaissance  would  die  before  it 
began,  lacking  the  infrastructure  to  support  the  propagation  of  knowledge.  But  modem  technology  was  on  the  horizon. 

The  mid  fifteenth  century  saw  the  convergence  of  several  unrelated  developments:  the  wine  press,  paper  production,  oil- 
based  ink,  block  printing,  and  finally,  movable  type.  (Mukerji  and  Jones,  1995).  On  a spring  morning  in  1452,  the 
headlines  declared,  "The  Scribe  is  Dead.  Long  Live  the  Printer!"  Print  technology  was  soon  to  become  the  affordance  of 
an  aspiring  middle  class. 

The  modern  age  arrived  with  the  first  edition  of  Gutenberg’s  Bible.  The  cost  of  book  production  fell  by  several  hundred 
fold.  A printer  could  reproduce  a thousand  copies  of  a book  in  the  time  it  took  a scribe  to  reproduce  four  copies  of  the 
same  text.  Within  a few  years  there  were  more  circulated  copies  of  Scripture  and  the  major  classics  than  had  existed 
previously  for  all  time.  In  the  coming  years,  the  Scriptures  and  the  classics  became  affordances,  not  for  spiritual  ends,  but 
as  a means  of  learning  how  to  read  and  write,  as  a means  of  mastering  reason,  logic  and  rhetoric  (McCluhan,  1962).  By 
1501,  there  were  a thousand  print  shops  throughout  Europe.  They  had  produced  two  million  copies  of  texts,  35,000  titles 
(Mukerji  and  Jones,  1995). 

The  affordance  of  archived  knowledge  burst  free  from  the  constrictive  grip  of  monks,  priests  and  courtiers.  The  power  of 
print  technology  was  in  the  hands  of  an  emerging  ruling  class.  The  printing  press  was  an  affordance  for  Martin  Luther  to 
assert  his  Ninety-five  Theses,  challenging  established  authority.  The  emerging  years  brought  the  Reformation,  the 
Enlightenment  and  democracy.  The  bourgeois  state  triumphed  over  monarchies.  Liberalism  instituted  basic  rights  and 
freedoms:  The  rights  of  private  property,  freedom  of  self-expression,  and  freedom  of  the  press.  These  "self-evident"  rights 
spread  throughout  the  Western  world.  They  were  the  expressed  freedoms  for  all  citizens,  but  they  were  particular 
effectivities  for  those  who  owned  property,  those  who  possessed  the  skills  associated  with  self  expression,  and  those 
who  owned  or  controlled  the  presses  (Bagdikian,  1993). 

The  modem  age  became  an  age  of  reading  and  writing  (mostly  reading).  Book  learning  changed  the  nature  of  basic 
education  throughout  the  West.  The  providers  of  education  (initially  the  church,  and  ultimately  the  state)  perceived  it  as 
an  institutional  affordance  to  foster  morality,  patriotism  and  citizenship.  Education  itself  was  primarily  moral  (the 
intention  of  the  institution)  and  secondarily  intellectual  (the  intention  of  the  learner).  Educational  technologies  (the 
textbooks,  the  lessons,  the  methods)  were  designed  to  carry  out  the  primary  intention.  The  Massachusetts  law  of  1647 
made  its  intentions  explicit:  to  disable  the  "project  of  that  old  deluder  Satan  to  keep  men  from  the  kno\ riedge  of  the 
Scriptures"  (Massachusetts  School  Law,  1647).  Doctrinal  intentions  prevailed  for  over  two  hundred  yea.*s  in  American 
Schools.  Instructional  technology  reflected  those  intentions.  The  hornbook  was  designed  to  teach  basic  reading,  but  more 
importantly,  Christian  doctrine.  This  technology  was  a wooden  frame  on  a handle.  The  frame  enclosed  a single  printed 
page  containing  the  English  alphabet  followed  by  the  text  of  the  Lord's  Prayer.  The  New  England  Primer  was 
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characterized  by  heavy  religious  platitudes:  "In  Adam’s  Fall  We  sinned  all."  ‘The  idle  Fool  Is  whipt  at  School/'  etc.. 
Noah  Webster's  Blueback  Speller  offered  short  parables  with  morals  explicitly  interpreted  on  each  page.  From  1836  to 
1907  the  McGuffy  Readers  set  the  standard  for  American  grammar  school  education.  There  is  rarely  a page  in  that  series 
of  six  readers  which  does  not  address  itself  to  some  moral  problem  from  the  standpoint  of  Protestant  Christianity  and 
Victorian  ethics.  The  six  books  propagated  a middle-class,  conventional,  paternalistic  morality,  reflecting  the  values  of 
the  dominant  culture  (Commanger,  H.S.,  1962).  Education  molded  the  collective  conscience.  Literacy  was  a means  for 
social  control. 

Education  in  the  twentieth  century  retains  a content-centered  paradigm  of  institutional  affordance  (Lemke,  1994). 

However,  this  has  been  a century  of  major  curricular  reforms.  The  changes  have  reflected  a more  diverse  and  complex 
society  with  a broader  range  of  values  and  priorities.  As  our  society  continues  to  diversify,  as  the  need  for  technical  skill 
increases,  as  the  world  changes  faster  than  textbooks  and  lesson  plans  can  keep  pace,  the  emphasis  on  curriculum  content 
is  giving  way  to  a focus  on  the  learning  process.  Constructivism,  considered  radicalism  a decade  ago,  is  the  dominant 
perspective  at  most  educational  conferences.  Content  objectives  are  giving  way  to  a focus  on  skills  development, 
instructional  designs  are  giving  way  to  learning  environments  where  students  define  the  central  tasks,  including  how  the 
learning  should  be  monitored,  assessed  and  adjusted  to  achieve  the  desired  outcomes.  Such  strategies  bring  flexibility  to 
the  educational  process,  adjusting  to  continued  change  in  a postmodern  world. 

Modern  to  Postmodern 

The  term  postmodern  is  often  over-used  and  ill-defined.  But  we  can’t  come  up  with  a better  term  to  describe  the 
transitions  taking  place  today  in  educational  technology.  What’s  the  difference  between  modem  and  postmodern?  A telling 
illustration  comes  from  Alluquere  Rosanne  Stone  (1995).  Stone  describes  an  encounter  between  former  quiz  kid  Charles 
Van  Doren  and  computer  interface  creator,  Brenda  Laurel.  The  encounter  took  place  at  Atari  Labs  in  the  late  1980s  where 
Dr.  Laurel  was  engaged  in  research  on  interactivity.  Van  Doren  was  working  for  the  Encyclopedia  Britannica  company 
which  had  begun  work  on  a new  interactive  version  on  CD  ROM.  Stone  describes  the  story: 

He  came  by  Laurel's  new  office  in  the  lab  one  day  and  proceeded  to 
chat  her  up  about  interactivity.  Laurel’s  ears  perked  up.  "That's 
great,"  she  said,  "I’m  working  on  interactivity  too." 

"You  are?"  Van  Doren  said. 

"Sure,"  Laurel  said  enthusiastically.  'Tve  got  this  idea  for  an 
interactive  educational  thing  about  whales  told  from  multiple 
perspectives  - whales  from  an  Inuit  perspective  and  then  whales 
from  a whaling  corporation  perspective,  and  a Greenpeace 
perspective,  say.  Multiple  narrative  thread,  user  selectable.  It’d 
fit  right  into  your  interactive  encyclopedia." 

Van  Doren  turned  red  and  began  to  make  a peculiar  noise.  After  a 
few  seconds  laurel  realized  he  was  sputtering.  Finally  he  burst 
into  speech.  "Encyclopedias  don't  present  viewpoints,"  he  said, 
biting  off  the  words.  "Encyclopedias  present  truth." 

This  encounter  is  a vivid  contrast  between  modem  and  postmodern  thinking.  The  modernist  presents  the  truth,  just  as  it 
was  nailed  on  the  door  at  the  church  of  Wittenburg  in  1557;  just  as  it  was  introduced  to  American  children  in  six 
complete  volumes  for  most  of  the  19th  Century,  just  as  it  was  represented  on  the  1950s  television  quiz  show  "the 
64,000  Question".  The  postmodernist  offers  multiple  perspectives  from  a complex  world,  including,  perhaps,  the 
modernist  conception  of  truth  (Wilson,  1994). 

Today’s  information  technology  enables  the  transition  from  modern  to  postmodern,  no  less  than  the  printing  press 
enabled  the  transition  from  pre-modem.  How  many  days  would  it  take  a scribe  to  provide  us  with  a copy  of  Aristotle's 
Rhetoric?  Seven  days?  Fifteen  days?Thirty  days?  And  what  is  the  cost  of  skilled  labor?  $10/hr?  $ 1 5/hr?  $25/hr?  At  the 
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most  conservative  estimate,  the  manuscript  would  cost  more  than  $500.  But  the  modem  printing  press  makes  it  available 
for  less  than  $10.  And  postmodern  electronic  publishing  provides  it  at  virtually  no  cost!  Our  computer  in  Denver  is 
connected  via  the  Internet  to  a computer  at  Virginia  Tech,  where  the  complete  works  of  Aristotle  reside  on  a rotating 
magnetic  disk.  We  can  download  Rhetoric  instantly,  at  no  charge. 

But  is  it  really  free?  There  must  be  a cost  associated. 

There  is  a cost  associated  with  disk  data  storage.  With  current  technology,  the  cost  is  about  fifty  cents  per  megabyte.  To 
store  Aristotle's  Rhetoric  (400  KB),  the  cost  is  about  twenty  cents. 

At  such  a bargain,  why  don't  I store  it  on  my  own  disk?  The  answer  is,  I don’t  need  to.  I can  get  it  anytime  I want  from 
Virginia  Tech  (or  one  of  the  forty  some  other  Internet  sites  that  have  it!)  What’s  more,  I can  link  it  (as  we  have  done 
above)  to  my  own  writing.  To  the  extent  that  such  capability  serves  our  intended  purpose,  the  link  is  an  affordance  for 
us,  the  authors.  To  the  extent  that  the  link  serves  the  reader’s  intentions  (regardless  of  our  intentions),  the  link  is  an 
affordance  for  the  reader. 

Our  point  is  that  technology  has  changed  the  way  we  consider  content.  In  a 14th  century  university  classroom,  a 
manuscript  of  Rhetoric  was  likely  the  exclusive  source  of  information.  That  was  a constraint  imposed  by  the  technology. 
Today,  we  can  download  Rhetoric  in  seconds,  but  we  can  also  download  related  texts,  commentaries,  research  studies, 
historical  perspectives,  ad  infinitum.  We  can  scan  these  texts  with  the  eye  of  a computer,  discovering  patterns,  linking 
them  to  allied  texts,  offering  multiple  dimensions  of  textual  analysis  (Olsen,  1993)  (Landow,  1994). 

Interactivity 

In  recent  years,  a prime  objective  in  media  development  has  been  the  quality  of  interactivity.  This  has  been  especially 
true  with  educational  media,  starting  in  the  1950s  with  Skinner's  ’’teaching  machine",  evolving  in  the  70s  to  "response- 
and-branch"  CBT,  progressing  into  the  ’80s  with  Apple's  ’’poke-and-see"  hypercard  and  ultimately,  "answer-vision",  the 
hopes  behind  interactive  video.  All  of  these  are  examples  of  closed  systems  in  which  the  content  is  pre-defined,  responses 
are  anticipated,  and  action  is  controlled  by  the  designer  alone.  Interactive  video  pioneer,  Andy  Lippmann,  lamented  the 
inherent  weakness  of  such  closed  systems.  None  seemed  to  satisfy  his  definition  of  interactivity,  the  mutual, 
autonomous,  and  simultaneous  activity  of  both  participants  working  toward  a common  goal.  Lippman  suggested  five 
criteria  which  are  necessary  to  satisfy  this  definition:  (Brand,  1987)  (Stone,  1995) 

* interruptability,  the  ability  of  either  participant  to  interrupt  the 
other  at  any  point. 

* graceful  degradation,  the  ability  to  set  aside  the  unanswerable 
questions  in  a way  that  does  not  halt  the  conversation. 

* limited  look-ahead,  the  quality  that  makes  it  impossible  to  predict  the 
ultimate  outcome  of  a conversation  by  either  party. 

* no  default,  a quality  which  allows  the  share  of  a conversation  to 
develop  naturally,  organically,  without  a preplanned  path. 

* the  impression  of  an  infinite  database,  the  quality  of  limitless 
choices  and  realistic  responses  to  any  possible  input. 

Lippman’s  criteria  are  fundamentally  unreachable  under  the  constraints  of  a closed  technology.  But  the  open  architecture 
of  the  Internet  transfers  interactivity  from  an  affordance  of  designer  control  to  that  of  user  control.  From  flat  textual 
domains  to  full  motion  video,  the  Internet  transforms  the  mechanistic  character  of  modern  media  into  a postmodern 
phenomenon,  shifting  agency  from  author  to  user.  The  designer  has  lost  control  (Kelly,  1995),  but  the  medium  has 
gained  credibility.  We  no  longer  have  to  contrive  interactive  "lessons”  and  exercises.  The  real  world  is  waiting  on  the 
other  side  of  the  terminal. 

When  we  speak  of  affordances  in  the  study  of  media,  we  are  cautioned  as  designers  to  resist  the  assumption  that  a given 
medium  affords  a specific  set  of  learning  outcomes.  In  reality,  learning  is  distributed  between  the  medium,  the  learner  and 
the  context  (Jonassen,  Campbell  & Davidson,  1994).  There  is  nothing  inherent  in  the  Internet  that  guarantees  learning. 
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But  in  a specific  context  involving  learning  activities,  such  as  research,  collaboration,  self  expression,  and  reflection,  the 
Internet  offers  multiple  affordances,  so  numerous  that  it  may  be  a mistake  for  us  to  treat  it  as  a medium.  It  is  really  an 
infrastructure  which  brings  together  media,  tools,  people,  places  and  information,  expanding  the  range  of  human 
capabilities. 

If  we  treat  the  Internet  as  an  infrastructure,  then  we  can  draw  comparisons  between  the  Internet  and  other  infrastructures 
which  society  has  come  to  rely  upon  for  learning  and  education.  These  include: 

* a library 

* a publishing  house 

* a school 

* a community 

* a city 

* (please  add  to  this  list) 

We  suggest  that  the  Internet  offers  multiple  affordances,  not  unlike  other  infrastructures  for  learning.  And  we  suggest  that 
the  Internet  overcomes  significant  factors  which  have  historically  constrained  the  others.  A comparative  analysis  may 
illustrate  our  point. 

Library 

The  library  is  an  infrastructure  containing  multiple  technologies  working  together  to  deliver  specific  information  in 
response  to  an  inquiry.  The  patron  consults  a data  base  (card  catalog  or  electronic  index)  to  find  a range  of  books  which 
may  contain  the  desired  information.  The  patron  selects  the  most  appropriate  listings  from  among  those  suggested,  then 
proceeds  to  the  locations  in  the  library  directed  by  each  selection.  For  each  book  that  is  located,  the  patron  consults  its 
index  and  table  of  contents,  then  leafs  though  the  pages  to  confirm  whether  the  book  holds  pertinent  information.  If  so, 
the  information  is  eventually  extracted  and  the  task  is  complete. 

Research  on  the  Internet  involves  a similar  process.  The  user  consults  a database  called  a search  engine.  A simple  or 
structured  query  will  yield  a set  of  hits  or  listings,  not  unlike  the  selections  from  a library  catalog.  The  user  selects  the 
most  appropriate  hits,  retrieving  each  document  electronically.  Within  seconds  each  target  document  is  retrieved.  A 
hypertext  index  or  table  of  contents  will  allow  the  user  to  dip  into  the  text  wherever  her  interest  leads.  For  plain  text 
documents,  the  user  can  invoke  a keyword  search  utility  on  her  browser  locating  specific  paragraphs  of  interest.  Pertinent 
information  is  extracted  and  the  task  is  complete. 

Like  a library,  the  Internet  affords  great  potential  for  research.  The  processes  of  information  searching  are  similar,  if  not 
identical.  The  results  are  not  so  similar.  At  the  present  time,  a good  research  library  is  likely  to  yield  a higher  quality 
result  than  the  Internet.  But  the  Internet  is  likely  to  yield  more  up-to-date  information  than  a library. 

A comparison  of  constraints  will  reveal  some  important  differences: 

Schedule 

A library  is  constrained  by  limited  hours  of  operation.  Serious 
researchers  are  painfully  familiar  with  the  signification  of 
flickering  lights  toward  the  end  of  the  library’s  service  day.  But 
the  Internet  is  not  constrained  by  schedule.  It  is  as  available  as 
a personal  computer  and  a modem. 

Location 

A library  has  a fixed  location.  Most  communities  have  at  least  cne 
library  but  rural  communities  are  especially  constrained.  The 
Internet  is  not  constrained  by  proximity.  It  is  accessible 
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anywhere  that  electronic  communication  is  possible.  Typically  the 
electronic  patron  accesses  Internet  resource  from  home  or  from  the 
work  place. 

Holdings 

A library's  holdings  are  constrained  by  past  acquisitions  and 
current  budgets.  Most  libraries  are  able  to  overcome  this 
constraint  by  interlibrary  loans.  The  Internet  is  one  huge  virtual 
library  whose  holdings  are  growing  daily.  At  the  present  time, 
resources  on  the  Internet  are  fundamentally  constrained  by 
copyright.  Most  books  in  print  are  available  in  book  stores  or  at 
the  library,  but  they  can  not  be  obtained  online.  This  is  not  a 
natural  constraint  of  the  technology,  but  the  result  of  lagging 
proprietary  laws  associated  with  prior  technologies  (Lehman, 

1994). 

Another  constraint,  at  least  for  the  present,  is  laborpower:  the 
effort  required  to  migrate  public  domain  literature  to  electronic 
substrates.  This  is  a noble  effort,  motivating  thousands  of 
volunteer  scribes  (see  Ryder,  1995),  but  at  this  moment  th^bulk 
of  the  world's  literature  remains  somewhere  on  a dusty  shelf. 

Accessibility 

A library  book  is  accessible  within  minutes,  hours,  days  or  weeks, 
depending  on  the  book's  location  and  availability.  Electronic 
texts  are  available  instantly.  Checked-out  and  overdue  documents 
are  unknown  in  the  virtual  library.  Research  sessions  need  not  be 
interrupted  for  lack  of  immediate  access.  What's  more, 
instantaneous  retrieval  enables  hypertext  (linking  between  texts) 
as  a research  affordance  — something  not  feasible  with 
conventional  print  media. 

Timeliness  of  holdings 

Libraries  receive  information  once  it  is  published.  Published 
information  is  usually  well  written,  edited  and  thoroughly 
reviewed  for  accuracy.  For  most  quality  books,  this  process  takes 
several  months.  New  material  finds  its  way  to  the  Internet 
immediately.  Many  articles  are  accessible  on  the  net  months  before 
they  become  published  in  print  form.  Authors  often  post  early 
drafts  of  their  work  for  purposes  of  feedback  and  peer  review 
(Hamad,  1995).  (This  paper  was  accessible  on  the  net  as  this  very 
sentence  was  being  drafted;) 

The  immediate  overriding  constraint  of  the  Internet  is  the  fact  that  few  users  have  learned  how  to  manipulate  Internet 
tools  with  the  same  confidence  that  they  can  manipulate  a library.  As  the  Internet  evolves,  we  can  observe  the  changes 
before  our  eyes.  In  the  past  two  years,  the  number  of  text  objects  on  the  World  Wide  Web  grew  from  several  thousand  to 
several  million.  The  tools  for  extracting  information  showed  their  limitations  as  Internet  holdings  constantly  increased. 
New  tools  have  been  laid  down  over  the  old,  smarter,  faster,  yielding  more  astounding  results.  Tools  are  evolving  in  the 
direction  of  object-oriented  simplicity.  But  users  are  hard-pressed  to  keep  up  with  the  changes. 
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The  implications  for  instruction  and  performance  technology  in  this  context  are  obvious.  Most  of  us  learned  how  to  use 
the  Library  when  we  were  in  school,  and  we  learned  by  doing  --  doing  research.  In  the  process  of  solving  research 
problems  we  had  the  scaffolding  of  librarians  and  the  coaching  of  good  teachers.  We  suggest  that  Internet  skills  deserve  a 
similar  treatment.  The  power  of  Internet  resources  remains  latent  to  those  without  the  skills  to  use  them. 

But  who  are  the  librarians  in  this  virtual  library?  Who  will  provide  the  scaffolding  and  coaching  for  the  unskilled 
researcher?  Who  will  undertake  the  task  of  conjoining  people  and  knowledge  (Birchall  and  Randa,  1995)?  Who  will 
classify  knowledge  and  information?  These  tasks  don’t  go  away  in  the  virtual  environment,  but  the  agency  of 
librarianship  shifts  from  the  center  to  the  periphery.  The  role  of  virtual  librarian  is  distributed.  In  the  virtual  library  there 
is  no  central  keeper  of  knowledge,  only  curators  of  particular  views  (Kelly,  1995).  The  role  of  organizing  and  classifying 
knowledge  is  passed  to  each  user.  A user  saves  and  organizes  the  information  that  is  personally  useful.  As  one's  interests 
and  knowledge  become  specialized,  a niche  emerges  for  a specific  field  of  knowledge.  The  ones  who  use  the  information 
are  the  ones  who  will  organize  and  classify  that  body  of  knowledge,  primarily  for  personal  use  but  also  to  serve  the 
virtual  community.  The  intellectual  labor  invested  in  research  is  freely  shared  on  the  net  as  users  make  public  the  artifacts 
of  their  own  knowledge  construction:  the  bookmarks,  lists  and  directories  of  information  we  compile  for  use  in  our  own 
intellectual  efforts.  One  person's  artifact  is  another's  affordance. 

In  my  own  work,  I maintain  a set  of  bookmarks  related  to  Instructional  Technology  (see 

http://www.cudenver.edu/~mryder/itcon.html).  While  the  work  is  a product  of  my  own  research  for  my  own  purposes,  it 
can  provide  scaffolding  for  others  with  similar  interests.  By  making  public  this  entailment,  I have  the  benefits  of 
interaction  with  others  who  are  drawn  to  the  resource  because  of  their  own  allied  modes  of  inquiry.  The  ethic  of  sharing 
information,  including  classifications  of  this  nature,  is  the  emerging  culture  of  the  net.  The  motivation  is  no  different 
than  the  traditional  practice  of  publishing  scholarly  work. 

Publishing 

The  publishing  infrastructure  has  been  the  primary  conduit  for  knowledge  in  the  modern  world.  It  requires  capital  and 
considerable  labor  to  disseminate  information  in  published  form.  Between  the  modern  author  and  the  reader  is  an 
infrastructure  which  includes  editors,  type  setters,  printers,  binders,  and  a complex  marketing  and  distribution 
organization.  For  a publisher  to  print  anything  today,  a business  case  must  guarantee  proper  return  on  the  investment.  At 
a time  when  society's  need  for  diverse  information  expands  geometrically,  the  number  of  corporations  which  control  the 
publishing  infrastructure  continues  to  decline  (Bagdikian,  1990).  While  freedom  of  the  press  is  extended  to  all,  its 
affordance  is  an  exclusive  privilege. 

The  concept  of  free  expression  is  changing  as  we  pass  from  the  modern  to  the  postmodern.  Having  the  right  to  free 
expression  is  something  entirely  different  than  having  the  affordance  of  expression.  Just  as  technology  challenged 
exclusive  control  of  knowledge  in  the  fifteenth  century,  exclusive  control  of  information  is  challenged  again  today,  this 
time  by  the  Internet.  On  the  Internet,  there  is  no  central  control:  no  editorial  board  to  determine  acceptability,  no 
screening  committee  to  determine  relevance,  no  financial  board  to  assess  marketability.  Each  author  is  her  own 
publisher.  Each  reader,  his  own  editor.  All  that  is  required  for  publication  on  the  World  Wide  Web  is  a personal 
computer,  a modem,  and  a user  account  on  an  Internet  server.  Students  and  faculty  enjoy  such  affordance  at  most  major 
universities.  Commercial  providers  offer  public  access  at  at  costs  comparable  to  telephone  rates.  Public  schools  are  on 
the  World  Wide  Web,  publishing  hypertext  productions  by  teachers  and  students.  Today  a third  grader  has  the  same 
affordance  for  expression  as  any  other  netizen,  be  they  politicians,  professors  or  published  authors.  What's  more,  her 
physical  youth  can  be  masked  by  her  abilities  of  expression,  making  her  a virtual  equal  among  adult  netizens  (Fishman 
& Pea,  1993). 

During  the  past  decade,  constructivist  researchers  have  stressed  the  importance  of  anchoring  instruction  to  genuine  tasks 
in  situated  contexts  (Cognition  and  Technology  Group  at  Vanderbilt,  1992)(Brown,  Collins,  and  Duguid,1989)(Resnick, 
1987)(Jonassen  and  Duffy,  1992).  The  Internet  offers  countless  affordances  toward  that  instructional  goal.  Consider  a 
seventh  grade  class  learning  English  grammar,  spelling  and  rhetoric.  Consider  the  World  Wide  Web  as  an  affordance. 
Imagine  the  possibility  of  our  students  providing  a web  page  that  is  a genuine  service  to  the  Internet  community, 
something  the  students  care  about,  and  something  that  is  needed.  Consider  a likely  niche,  such  as  a weekly  review  of 
children's  television,  a computer  help  system  for  low-tech  parents,  a FAQ  database  on  snow  boarding,  roller 
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blading,  etc..  Writing  for  an  audience  of  peers  around  the  world  is  likely  to  elicit  motivations  that  go  beyond  teacher 
expectations.  To  the  extent  that  students  use  the  opportunity  to  present  themselves  before  a world  wide  audience  their 
writing  will  be  influenced  by  motivations  of  ethos,  style,  and  delivery:  the  elements  of  classical  rhetoric.  The 
constraining  factors  are  likely  to  include  the  teacher's  comfort  level  with  the  technology,  and  her  willingness  to  yield 
control  of  the  resource  to  her  students,  the  level  of  her  perception  about  the  Internet  as  ai  affordance  for  learning  goals 
(not  necessarily  for  other  institutional  goals). 

The  affordance  of  world  wide  public  expression  is  something  new  for  all  of  us.  With  the  Internet,  the  perceived  capability 
of  self  expression  in  a world-wide  forum  can  be  extended  realistically  to  the  common  person.  However,  there  is  no 
expectation  that  each  person  will  embrace  it.  Consider  Socrates'  reaction  to  literacy.  Consider  the  skepticism  of  most 
Americans,  saturated  with  the  hype  of  technology  and  media.  By  definition,  an  affordance  is  the  the  potential  for  action. 
This  implies  that  there  is  a will  toward  specific  action.  Implied  in  the  affordance  is  the  intentional  desire  for  action  and 
the  recognition  that  the  object  can  enable  one's  intentions(4). 

Quality  Control 

For  the  reader,  a fundamental  constraint  has  to  do  with  the  accuracy,  veracity  and  reliability  of  online  information.  This  is 
not  to  say  that  accurate  information  can't  be  found  on  the  net.  The  implication  has  more  to  do  with  the  shifting  role  of 
the  reader  in  assessing  the  accuracy,  validity  and  applicability  of  materials  which  were  heretofore  left  to  editors,  experts 
and  peer  professionals  (Ryder,  1995),  (Eco,  1979),  (Landow,  1994),  (Kelly,  1995).  An  additional  skill  will  be  essential  to 
our  basic  education-  the  skill  of  abductive  logic  (Peirce,  1902)  (Shank,  1994).  The  goal  in  abductive  logic  is  not 
ultimate  truth.  It  is  the  ability  to  advance  one's  inquiry.  Abductive  logic  is  the  logic  of  signs:  the  ability  to  extract 
meaning  from  a given  set  of  circumstances  and  then  to  adjust  one's  inquiry  as  new  information  unfolds.  The  skill  of 
Sherlock  Holmes  is  critical  skill  of  the  information  age. 

There  are  Internet  tools  to  assist  in  this  process.  They  are  called  robots  or  agents.  An  agent  is  a software  tool  that  is 
dispatched  to  accomplish  a specific  task.  As  information  continues  to  abound  on  the  Internet,  we  will  need  such  tools  to 
assist  us  in  our  searches.  The  standard  tools  are  search  engines  which  accept  a set  of  keywords  from  a user  and  return  a 
list  of  resources  in  rank  order,  listing  first  those  which  best  match  the  search  criteria.  The  problem  is  that  searching  on  a 
single  word  can  yield  overwhelming  and  debilitating  results  (Eco,  1995).  Stone  (1995)  suggests  that  the  ideal  model  for 
an  agent  is  the  graduate  student.  Graduate  students  are  trained  to  do  research.  They  can  assess  the  usefulness  of  new 
information  within  in  the  context  of  the  present  inquiry.  A research  assistant  is  attuned  to  a professor’s  work,  and  is  able 
1.  gather  the  kind  of  information  that  fits  a given  research  project.  Can  software  agents  be  designed  to  fit  that  model?  So- 
called  intelligent  search  applications  are  available  today.  Instead  of  submitting  a simple  keyword,  phrase  or  boolean 
expression,  the  user  can  submit  a complex  object:  a bibliography,  a list,  an  index,  an  article.  The  intelligent  agent 
discards  incidental  and  common  words  from  the  text  object,  prioritizes  the  remaining  words  by  the  frequency  of  use. 
Multiple  searches  ensue,  and  results  are  evaluated  against  the  model  object.  Tools  of  this  nature  produce  quality  results 
for  minimal  time  investment,  overcoming  the  debilitating  constraints  of  too  much  information  associated  with  simple 
searches. 

The  fundamental  characteristic  of  the  Internet  is  that  it  cannot  be  controlled.  While  local  legislation  can  influence  pockets 
of  control,  the  Internet  is  a world  wide  phenomenon,  with  no  one  in  charge,  (with  everyone  in  charge).  The  Internet  is 
not  the  McGuffy  Reader.  In  more  modem  times  educators  could  easily  control  what  their  students  read  and  direct  how 
those  tender  minds  might  be  shaped.  The  educator's  role  was  largely  in  the  selection  of  good  and  proper  literature, 
selecting  resources  which  combined  intellectual  development  with  moral  edification.  The  learner's  moral  edification  was 
directed  externally  by  a teacher,  by  an  editorial  committee,  or  by  a school  board. 

Today  the  classics  of  literature,  fine  essays  and  poetry  still  abound.  They  are  rapidly  finding  their  way  online.  There  will 
be  a time  in  the  next  ten  or  fifteen  years  when  every  essay  or  poem  that  was  ever  published  by  McGuffy  or  any  other 
reputable  source  will  be  available  in  digital  format  for  readers  who  wish  to  partake  of  these  edifying  materials.  But  along 
side  such  uplifting  literature  will  be  objectionable  material,  countless  resources  of  questionable  value,  some  resources  of 
objectionable  content.  The  supreme  difficulty  in  the  Information  Age  will  be  the  ability  to  restrict  a learner's  education  to 
a "sanctioned"  body  of  literature.  The  discipline  required  of  learners  in  the  postmodern  age  goes  beyond  that  of  all 
previous  generations.  The  role  of  education  in  the  age  of  information  will  be  the  development  of  disciplined  readers. 


skilled  in  the  art  of  abductive  logic.  Since  we  can  no  longer  filter  and  select  proper  materials  for  our  students,  our 
highest  calling  as  educators  will  be  to  support  students  in  developing  such  discipline  for  themselves. 

Whitehead  (1929)  explained  to  an  earlier  generation  of  teachers  that,  "discipline  is  knowledge  in  the  presence  of  freedom". 
Giving  students  the  freedom  to  choose,  then  supporting  them  through  the  anguish  of  moral  and  logical  choices,  this  is 
practice  that  builds  discipline,  skill,  and  moral  fortitude.  It  will  be  the  highest,  most  valued  instructional  model  for  the 
next  few  decades. 

Summary 

In  contemplating  what  the  future  might  hold  for  education  it  is  sometimes  useful  to  gaze  into  the  past.  We  have 
attempted  to  show  how  formal  knowledge  broke  free  from  cloistered  protection  to  broad  dissemination  in  the  age  of 
Guttenberg.  Knowledge  in  the  the  modem  period  followed  logocentric  patterns  with  one-to-many  relationships.  The 
postmodern  period  promises  yet  greater  dissemination  marked  by  a many-to-many  relationships.  The  developments 
brought  about  by  technology  allow  for  new  possibilities.  We  are  seeing  trends  in  education  toward  distributed, 
collaborative  models  of  learning.  Agency  is  shifting  from  center  to  periphery,  from  teacher  to  learner,  from  author  to 
reader,  from  librarian  to  researcher,  from  curriculum  to  context.  The  affordances  of  public  expression  have  exploded, 
allowing  any  school  child  to  represent  herself  before  a world-wide  community  of  learners  (5).  The  affordances  of 
information  retrieval  have  advanced  to  the  point  where  a simple  query  can  yield  a flood  of  information. 

This  is  where  the  modem  and  postmodern  minds  collide.  To  the  modem  mind,  there  is  too  much  information.  The  world 
is  exploding  with  ideas  and  perspectives  that  cannot  possibly  be  consumed.  We  must  control  what  people  read  so  that 
truth  might  prevail  over  misinformation,  so  that  quality  might  prevail  over  mediocrity,  so  that  correct  ideas  might 
prevail  over  anarchy.  But  to  the  postmodern  mind,  attempts  to  control  information  are  futile  and  naive.  What  control 
exists  in  the  postmodern  world  will  emerge  - not  from  the  center,  but  from  the  periphery.  The  genie  is  out  of  the  bottle. 
There  is  no  chance  of  forcing  him  back.  Modem  educators  have  learned  how  to  utilize  constraints  of  the  environment  as 
a means  of  scaffolding  in  the  instructional  process.  Constraints  can  limit  a field  of  information,  making  it  more 
manageable  for  learners.  Constraints  of  grammar  and  syntax  are  necessary  for  language  acquisition.  General  social 
behavior  is  learned  from  constraints  involving  etiquette  and  protocol.  We  have  shown  that  the  Internet  overcomes  many 
of  the  constraints  imposed  by  traditional  educational  infrastructures.  But  the  freedoms  that  suddenly  emerge  are 
frightening.  The  modem  educator  might  be  tempted  to  impose  artificial  constraints  on  the  medium  in  order  to  control  and 
manage  the  educational  environment.  Software  is  available  at  this  moment  which  allows  a teacher  or  parent  to  filter 
information  and  restrict  access  to  objectionable  sources  on  the  net.  We  acknowledge  that  such  artificially  imposed 
constraints  may  be  necessary  in  the  short  run  to  mitigate  controversial  dilemmas  and  situations  that  challenge  traditional 
thinking.  But  we  are  persuaded  that  the  real  challenge  for  education  is  to  discover  the  natural  constraints  associated  with  a 
highly  connected  and  deeply  fragmented  world.  A set  of  rules  or  policy  guidebook  for  the  postmodern  will  not  be  found. 
But  out  of  distributed  knowledge,  we  are  seeing  the  emergence  of  a new  ethic  with  an  entailing  structure  of  distributed 
control.  The  challenge  for  the  postmodern  educator  is  to  discover  the  capabilities  and  natural  constraints  associated  with 
distributed  pedagogy  for  scaffolding  learners  in  the  age  of  information. 

This  paper  is  available  in  hypertext: 

http://www.cudenver.edu/-mryder/aect_96.html 

Notes 

(1)  Given  the  fact  that  Socrates*  words  placed  him  in  disfavor  with  the 
authorities  of  the  time,  his  reluctance  to  commit  those  words  to  writing  is 
understandable. 

(2)  Oddly,  that  mode  of  information  transfer  prevails  in  universities  to 
this  day. 


(3)  Bookleaming  was  obviously  not  the  only  form  of  education.  The  most 
common  mode  of  education  in  the  middle  ages  was  neither  literate  nor  oral, 
it  was  situated  in  everyday  practice  (Lave  and  Wenger,  1991). 

(4)  There  is,  perhaps,  another  constraining  factor  associated  with  a newly 
gained  freedom:  fear.  In  his  study  of  education  among  third  world  peoples, 

Paulo  Freire  (1993)  observed  a natural  reluctance  among  prior  victims  of 
social  oppression  to  embrace  newly  gained  freedoms.  Freiere  describes  a 
confusing  behavior  among  many  enfranchised  citizens  as  they  enter  the 
historical  process  as  responsible  participants  in  society.  That  fear  is 
often  camouflaged  in  the  form  of  incredulous  defense  of  the  old  order. 

(5)  A Department  of  Education  survey  shows  the  number  of  schools  linked  to 
the  Internet  in  early  1996  is  approaching  50%  and  the  number  of  schools 
involved  in  accessing  the  Internet  is  up  35%  from  the  previous  year.  (Miami 
Herald  19  Feb  96  p25) 
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Abstract 

The  purpose  of  this  study  was  to  investigate  the  achievement  and  attitudinal  effects  of  learner  patterns  of  navigational 
behaviors  and  option  choices  in  using  a hypermedia-based  computer-assisted  instruction  (CAI)  unit  designed  for  college 
students.  Also  investigated  were  the  effects  of  two  incentive  structures  on  achievement  and  attitudes,  as  well  as  the 
relationships  among  prior  interest,  experience,  achievement  and  attitudes.  Students  were  randomly  assigned  to  either  a 
task-incentive  or  performance-incentive  version  of  a free-access,  HyperCard  format  instructional  computer  program. 

Multiple  regression  analysis  revealed  that  total  time  spent  on  instruction  and  practice  and  the  number  of  times  previous 
and  back  selections  were  made  were  significant  predictors  of  achievement.  Type  of  incentive  did  not  yield  a significant 
difference  in  posttest  scores  or  attitudes  between  treatments.  Overall,  subjects  had  positive  attitudes  toward  the 
instructional  program.  A positive  relationship  was  found  between  prior  experience  with  CAI  and  achievement.  The 
results  of  this  study  suggest  that  college  students  may  be  able  to  accurately  judge  in  what  fashion  they  need  to  review 
previously  viewed  screens  in  computer  programs  in  order  to  receive  higher  posttest  scores. 

Introduction 

Learner  control  in  computer-assisted  instruction  refers  to  the  control  of  options  within  programs  (Hicken, 
Sullivan,  & Klein,  1992).  Despite  the  attractiveness  and  cognitive  appeal  of  learner-centered  instruction,  results  of 
research  on  learner  control  to  date  have  been  inconclusive.  For  example,  when  given  control  over  content  review  (Kinzie, 
Sullivan,  & Berdel,  1988),  pacing  (Gray,  1987),  sequencing  (Steinberg,  1989),  and  presentation  medium  {Ross, 

Morrison,  & Odell,  1989)  students  gained  higher  achievement  scores.  However  many  studies  have  indicated  no 
significant  differences  between  learner  control  or  program  control  (Klein  & Keller,  1989)  or  poorer  learning  when 
students  control  their  own  learning  (Fry,  1972).  As  a result,  Hannafm  (1984)  and  others  have  called  for  caution  in  giving 
students  control  over  CAI  programs. 

Based  on  the  work  of  Carrier  and  her  colleagues  (cf.  Carrier,  Davidson,  & Williams,  19$5;  Carrier,  Davidson, 
Williams  & Kalweit,  1986)  virtually  all  of  the  more  recent  learner-control  research  has  been  conducted  by  providing 
students  with  programs  in  which  they  either  add  or  by  pass  instructional  elements.  Researchers  have  begun  to  use 
programs,  called  Full-Minus  or  Lean-Plus,  in  which  students  can  add  or  bypass  examples,  practice  items  and  review 
(Hicken,  Sullivan,  & Klein,  1990;  Igoe,  1993;  Hannafm,  1993).  Results  of  these  studies  have  been  inconclusive  with 
regard  to  the  effect  of  learner  control  on  achievement. 

More  consistently  favorable  results  have  been  obtained  in  studies  correlating  student  attitudes  and  learner  control. 
Pascal  (1971)  found  that  students  who  could  choose  the  format  in  which  instruction  was  presented  enjoyed  the  subject 
matter  more.  Fisher,  Blackwell,  Garcia  and  Greene  (1975)  reported  that  students  who  could  choose  the  difficulty  and 
amount  of  math  practice  became  more  engaged  in  the  instruction  than  those  who  could  not.  Morrison,  Ross,  and 
Baldwin  ( 1 992)  found  that  learners  allowed  to  choose  the  amount  and  context  of  practice  problem  had  more  positive 
attitudes  than  learners  in  a program-controlled  treatment. 

A factor  that  has  not  been  widely  studied  in  learner-control  studies  is  incentive  to  perform.  Tennyson  and 
Buttrey  (1980)  observed  that  the  use  of  an  actual  classroom-related  incentive,  that  is,  a grade,  was  missing  from  most 
learner-control  studies.  Whereas  a grade  would  have  provided  a performance-contingent  incentive,  many  studies  have 
utilized  only  a task-contingent  incentive,  in  which  simple  completion  of  the  task  constituted  student  performance.  In 
comparing  task-contingent  incentives  and  performance-contingent  incentives,  Hicken  et  al.  (1992)  found  that  the 
performance-contingent  incentive  group  performed  better  on  the  posttest.  When  comparing  types  of  performance- 
contingent  incentives,  Igoe  (1993)  found  that  there  were  no  significant  differences  between  students  who  selected  the  grade 
they  wanted  to  achieve  and  those  who  were  assigned  a specific  achievement  level.  Hicken  et  al.  (1992)  advised  that 
further  research  on  incentives  in  learner-control  studies  should  be  conducted. 

Many  of  the  instructional  programs  created  for  learner-control  studies  have  used  hypermedia  technologies  such  as 
HyperCard  and  Toolbook.  In  general,  these  programs  have  been  primarily  linear  in  format  (Freitag  & Sullivan,  1995; 
Schnackenberg,  Sullivan,  Leader,  & Jones,  1995).  They  have  not  allowed  learners  to  go  back  at  any  time  to  previous 
instruction  or  to  branch  to  other  material  within  the  program.  Little  research  has  been  conducted  in  which  the  learners 
have  been  allowed  to  utilize  the  network-structure  of  hypermedia.  This  network-structure  frees  learners  to  explore 
instructional  programs  using  any  options  and  navigational  patterns  they  desire. 

Several  studies  investigating  the  use  of  navigational  aids  in  non-lincar  programs  have  recently  utilized  this 
network  structure.  When  comparing  a navigation  system  that  involved  jumping  to  and  from  an  index  separate  from  tr-t 
displays  to  one  that  presented  navigation  options  directly  on  the  text  pages,  Wright  and  Likorish  (1990)  found  that  acait 
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subjects  preferred  index  navigation  in  a book-like  hypertext  program  and  page  navigation  for  information  that  was  in  the 
form  of  a matrix.  Stanton  and  Stammers  (1990)  reported  that  adults  who  were  trained  using  a non-linear  program  were 
able  to  carry  out  a simulated  task  significantly  better  than  those  who  were  trained  using  a linear  program.  Additionally 
when  comparing  the  effectiveness  of  using  analytical  approaches  or  browsing  techniques  to  find  information  in  a 
hypertext  environment,  Marchionini  and  Shneiderman  (1988)  found  no  significant  differences  among  high-school  age 
students. 

The  purpose  of  the  present  study  was  to  investigate  the  achievement  and  attitudinal  effects  of  learner  patterns  of 
navigational  behaviors  and  option  selections  while  they  used  a hypermedia-based  instruction  unit  designed  for  college 
students.  Also  investigated  were  the  effects  of  varying  incentive  structures  on  achievement  and  attitudes,  as  well  the 
relationships  among  prior  interest,  experience,  learner  achievement  and  attitudes.  Finally,  qualitative  data  regarding 
students*  perceptions  of  how  they  used  the  program  and  other  attitudinal  data  were  collected. 

Method 


Subjects 

Participants  in  the  study  were  1 10  undergraduate  students,  mostly  education  majors,  enrolled  in  an  introductory 
computer  course  at  a large  southwestern  university.  The  study  took  place  late  in  the  semester,  therefore  students  had 
already  used  IBM-PC  computers  to  learn  about  DOS,  word  processing,  databases,  and  spreadsheets.  Students  were  told 
that  this  section  of  the  course  would  introduce  them  to  using  Macintosh  computers  and  to  the  concept  of  “CAT\  that  is 
computer-assisted  instruction  for  education.  The  instructional  piogram  constituted  a regular  assignment  for  the  course, 
for  which  they  would  earn  points  toward  their  grade. 

Materials 

The  instructional  program  was  modified  from  one  created  by  Robert  Lievens.  It  was  designed  to  teach  various 
facts  about  the  prehistoric  Anasazi  people  of  the  American  southwest  in  a computer-assisted,  Macintosh-based  HyperCard 
format.  A computer-based  data  collection  program  (Dwyer  & Leader,  1994)  was  used  to  record  en-route  performance, 
option-selection  and  navigational  patterns,  time-on-task  data,  time-in-program  data,  attitude  data  and  posttest  data. 

For  the  study,  the  content  was  divided  into  the  five  units,  which  taught  nine  objectives,  as  follows: 

A.  Who  Were  the  Anasazi? 

1.  Identify  the  Anasazi,  prehistoric  Native  Americans  of  the  Southwest,  from  given  descriptions. 

B.  Where  Did  They  Live? 

2.  Locate  where  the  Anasazi  lived  on  a map.  ^ 

C.  What  Happened  to  the  Anasazi? 

3.  Identify  the  names  of  the  two  major  phases  of  Anasazi  development. 

4.  Given  a description  Oi  key  events  in  Anasazi  history,  identify  the  name  of  the  period  of  development. 

5.  Identify  what  most  agree  happened  to  the  Anasazi. 

D.  How  Did  the  Anasazi  Live? 

6.  Identify  the  type  of  villages  the  Anasazi  lived  in  at  the  height  of  their  culture. 

7.  Identify  features  common  to  their  homes,  villages  and  lives. 

8.  Identify  kivas  in  pictures  of  villages,  and  identify  their  use. 

E.  What  Did  They  Use  to  Make  a Living? 

9.  Identify  the  uses  of  types  of  tools  the  Anasazi  used  around  their  homes,  and  for  hunting. 

Each  unit  included  instructional  information  and  selected-response  practice  items.  Information  was  presented  in 
the  form  of  text,  a reference  map,  a timeline,  and  graphic  icons  that  could  be  clicked  on  to  obtain  facts  about  the  icons. 
Sounds  could  also  be  accessed  in  conjunction  with  clicking  on  some  of  the  graphics. 
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Students  navigated  through  the  program  by  using  a mouse  tool  and  selecting  buttons,  typically  presented  at  the 
bottom  of  each  screen.  On  the  first  screen  of  each  unit,  students  could  choose  to  review  the  objectives  for  that  unit. 
Students  could  at  any  time  choose  to  go  the  "Main  Menu".  The  "Main  Menu"  button  gave  students  free  access  to  move 
anywhere  in  the  program  at  any  time.  Students  could  move  to  different  topics,  the  practice  sets,  objective  screens,  or  the 
posttest  by  selecting  these  options.  They  could  also  skip  portions  of  instruction  or  practice  items  by  going  back  to  the 
menu  or  directly  to  the  practice  sets  or  quizzes. 

Each  screen  also  contained  arrow  buttons  in  which  the  students  could  return  to  the  card  they  just  viewed,  the 
previous  card  in  the  stack,  or  continue  to  another  card  in  the  program.  Students  could  quit  the  program  at  any  point  and 
later  to  resume  instruction  where  they  finished  by  clicking  the  "Quit"  button. 

In  addition  to  the  instructional  content,  the  CAI  program  included  introductory  instructions  for  using  the 
program  and  pre-  and  post-instructional  attitude  questionnaires.  The  four  item  pre-instructional  questionnaire  was 
designed  to  measure  students’  prior  interest  and  prior  experience  with  computers  and  CAI  as  well  as  preference  for 
learning  by  lecture,  video  or  pictures,  or  computers.  The  twenty -one  item  post- instructional  questionnaire  measured 
students'  attitudes  toward  the  program  content,  ease  of  use,  format,  and  clarity,  as  well  as  perceived  effort  and  motivation 
toward  the  topic.  Additional  data  were  collected  on  student  attitudes  and  use-patterns  through  a "notebook"  feature  built 
into  the  software.  Every  fifteen  minutes,  the  program  paused  and  requested  students  to  write  comments  about  how  they 
were  using  the  program.  Patterns  of  navigational  behaviors,  option  selection  and  time  were  tracked  by  a built-in 
computer  program  which  recorded  every  keypress  from  the  students  and  which  was  developed  to  track  patterns  thought  to 
enhance  learning,  in  particular  branching  back  and  forth  among  instruction  and  practice  activities. 

Two  versions  of  the  program,  differing  only  in  their  “incentive”  directions  in  the  introduction,  were  used  for  the 
study.  Instructional  presentation,  practice,  feedback,  and  the  questionnaires  and  posttest  were  identical  in  both  versions. 
Each  version  was  worth  a total  of  25  possible  course  points.  Students  in  the  “task-incentive”  treatment  were  informed 
that  they  received  all  25  course  points  for  simply  completing  the  program,  regardless  of  their  posttest  score.  Students  in 
the  “performance-incentive”  treatment  were  told  that  they  would  earn  points  based  on  their  scores  on  the  program 
posttest;  they  would  earn  25  points  if  they  scored  at  least  70%  on  the  posttest. 

Procedures 

Prior  to  receiving  a program  disk,  students  were  given  a ninety-minute  presentation  about  CAI  and  a 
demonstration  about  how  to  use  a Macintosh  computer.  Hands-on  experience  was  not  included  in  the  presentation, 
however  the  demonstration  included  large-screen  projected  computer  screen  images  and  verbal  explanations.  The 
instructional  program  itself  was  integrated  into  the  course  as  a regular  assignment  designed  to  introduce  the  students  to 
computer-assisted  instruction. 

After  the  presentation,  each  student  was  given  a disk  at  random  representing  one  of  the  two  versions  of  the 
program.  Separate  instruction  sheets  which  gave  directions  for  using  any  of  the  campus  computers  were  included  with 
the  disks.  Students  were  instructed  that  they  would  have  two  weeks  to  complete  this  assignment,  and  would  be  given 
one  lab  day  during  one  of  the  class  sessions  in  which  to  complete  it.  Several  experimenters  and  the  course  instructor 
were  available  at  various  computer  sites  to  answer  any  questions  about  the  program.  Program  disks  were  collected  two 
weeks  later  in  the  next  regular  class  session. 

Criterion  Measures 

The  program  included  a 20-item  multiple-choice  posttest  which  measured  achievement  on  the  objectives  of  the 
instructional  program. 

The  four-item  pre-instructional  questionnaire  measured  students’  prior  interest,  prior  computer  and  CAI 
experience,  and  attitudes  toward  computers.  The  twenty-one  item  post-instructional  attitude  questionnaire  assessed 
students’  satisfaction  with  the  material,  perceived  effort,  desire  for  more  information,  and  continuing  motivation.  (The 
items  included  in  these  two  instruments  are  included  in  Tables  2 and  3.) 

Design  and  Data  Analysis 

Analysis  of  variance  was  used  to  analyze  achievement  and  attitudinal  effects  of  the  two  incentive  structures. 
Backward  multiple  regression  analysis  was  performed  on  the  patterns  of  navigational  behaviors  and  time  as  related  to 
achievement.  Chi-square  analysis  was  used  to  determine  the  relationships  among  prior  interest,  experience,  learning 
achievement  and  attitudes.  Students’  perceptions  of  how  they  were  using  the  program  were  categorized  and  frequencies 
tabulated. 
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Results 


Achievement 

Students*  posttest  scores  were  relatively  high,  as  shown  in  Table  1.  The  mean  overall  score  was  17.26  out  of 
20,  with  the  range  of  scores  being  from  1 1 to  20.  Students  in  the  performance  incentive  treatment  (M  = 17.36)  did 
slightly  better  than  those  in  the  task  incentive  treatment  (M  = 17.16),  but  this  difference  was  not  significant. 


Table  1 

Posttest  Scores  by  Incentive  Treatment 


Treatment 

Mean 

SD 

N 

Performance  incentive 

17.36 

1.81 

55 

Task  incentive 

17.16 

2.07 

55 

Total 

JT 

17.26 

1.94 

110 

Note:  Posttest  scores  ranged  from  1 1 to  20  of  20  possible. 


Attitudes 

Pre-instructional  Attitudes.  Students*  responses  to  the  items  on  the  pre-instructional  questionnaire  are  shown  in 
Table  2.  As  can  be  seen,  students  were  relatively  neutral  (2.38  on  a scale  from  l to  5)  in  their  prior  interest  in  the  topic 
of  early  Native  Americans  in  the  Southwest.  The  two  “experience**  items  used  a scale  of  from  1 (a  lot)  to  3 (none/not  at 
all).  Students  indicated  they  had  little  prior  experience  with  the  Macintosh  computer  (M  = 2.15),  and  even  less 
experience  with  CAI  (M  = 2.3 1 ).  Students*  mean  score  of  1 .98  on  the  learning  mode  preference  item  indicated  some 
preference  for  learning  by  computers  over  lecture  or  video. 


Table  2 

Pre-instructional  Attitude  Data 


Ouestion 

Mean 

SP 

1.  Interest  in  topic 

2.38 

.98 

2.  Mac  experience 

2.15 

.66 

3.  Prior  CAI  experience 

2.31 

.66 

4.  Learning  mode  preference 

1.98 

.73 

Note:  N = 1 10.  Q1  scores  ranged  from  1 (high)  to  5 (low).  Q2  and  Q3  scores  ranged  from  1 (a  lot)  to  3 (none/not  at 
all).  Q4  scores  were  1 (lecture),  2(computers),  and  3 (video/media). 


Post-instructional  Attitudes.  Mean  scores  on  the  21  items  on  the  post-instructional  attitude  questionnaire  are 
shown  in  Table  3.  Scores  ranged  from  1 (Strongly  Agree)  to  5 (Strongly  disagree).  As  can  be  seen,  students*  attitudes 
were  generally  positive,  with  all  mean  scores  being  more  positive  than  3,  or  neutral.  Students  indicated  somewhat 
stronger  positive  responses  on  items  related  to  how  easy  the  program  was  to  use,  such  as  “I  knew  where  to  click**  (M  = 
1.55)  and  “It  was  easy  to  more  around  in  the  program*’  (M  = 1.76).  Students  also  indicated  they  felt  the  program  was 
well-designed,  for  instance,  “The  objectives  in  this  clearly  describe  the  content  and  practice**  (M  = 1-74),  “The  practice 
helped  me  to  learn  the  content**  (M  = 1.65),  and  ‘The  content  in  Anasazi  was  easy  to  understand**  (M  = 1.58).  Students 
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indicated  that  they  liked  using  the  computer  to  learn  this  content,  for  example,  ‘The  computer  program  was  a good  way 
to  learn  about  the  Anasazi”  (M  = 1 .64),  “I  like  learning  with  the  computer”  (M  - 1 .77),  “If  I had  the  opportunity,  I 
would  choose  to  learn  about  another  topic  using  a computer  program  like  this  one”  (M  - 197),  and  “I  would  want  to  use 
another  program  like  this  one  to  learn  about  another  subject”  (M  = 1 .79).  Not  surprisingly,  in  that  this  was  a computer 
course  mainly  for  education  majors,  students  were  somewhat  less  positive,  or  more  neutral,  in  their  attitudes  about  the 
Anasazi  content,  for  instance,  “The  content  was  interesting”  (M  = 2.04),  “I  am  interested  in  learning  more  about  the  early 
Native  Americans  of  the  Southwest”  (M  = 2.54),  and  “If  I had  the  opportunity,  I would  choose  to  learn  more  about  the 
Anasazi”  (M  = 2.57).  Finally  students’  mean  attitude  scores  indicated  different  amounts  of  enjoyment  and  satisfaction 
with  the  five  different  units.  The  unit  students  enjoyed  the  most  was  “How  Did  the  Anasazi  Live”  (M  = 1 92)  which 
included  information  about  houses  and  villages  and  many  sounds,  followed  by  “What  Did  They  Use  to  Make  a Living?” 
which  included  many  icons  of  tools  which  accessed  information  about  each  (M  = 2.05).  Next  students  enjoyed  “What 
Happened  to  the  Anasazi?”  (M  = 2. 17),  based  on  the  timeline,  and  finally  “Who  Were  the  Anasazi”  and  “Where  Did  They 
Live?”  (M  = 2.05).  Students  indicated  an  almost  equally  positive  perception  about  how  effectively  each  of  the  units  was 
presented  with  mean  scores  ranging  from  1.81  to  1 .86. 

Students*  post-instructional  attitude  mean  scores  overall  were  subjected  to  ANOVA  to  determine  any  incentive 
treatment  effects  on  overall  attitudes.  Results  on  these  overall  mean  scores  were  40.60  for  performance  incentive  and 
40.42  for  task  incentive;  these  differences  were  not  significant. 

Results  of  correlational  analyses  with  attitude  data.  Chi-square  correlational  analysis  among  the  pre- 
instructional  attitude  scores  and  posttest  scores  indicated  only  one  significant  positive  relationship  (p<.05),  which  was  for 
prior  experience  with  CAI.  A similar  relationship  appeared  in  the  results  of  the  chi-square  analysis  among  the  post- 
instructional  attitude  scores  and  posttest  scores.  A significant  positive  relationship  was  also  found  between  desire  to 
learn  other  content  using  CAI  and  posttest  scores. 


Table  3 

Post-instructional  Attitude  Data 


Ouestion 

Mean 

SD 

1 . The  objectives  in  this  program  clearly  described  the  content  & practice. 

1.74 

.60 

2. 1 used  the  objectives  to  guide  me  in  the  program. 

2.15 

.86 

3.  The  content  in  Anasazi  was  easy  to  understand. 

1.58 

.61 

4.  The  content  was  interesting. 

2.04 

.78 

5.  The  practice  helped  me  to  learn  the  content. 

1.65 

.74 

6.  It  was  easy  to  move  around  in  the  program. 

1.76 

.83 

7. 1 knew  where  to  click. 

1.55 

.72 

8.  The  computer  program  was  a good  way  to  learn  about  the  Anasazi. 

1.64 

.73 

9. 1 would  want  to  use  another  program  like  this  one  to  learn 
about  another  subject. 

1.79 

.94 

10. 1 like  learning  with  a computer. 

1.77 

.76 
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1 1. 1 am  interested  in  learning  more  about  the  early  native 
Americans  of  the  Southwest. 

2.54 

.92 

12.  If  I had  the  opportunity,  I would  choose  to  learn  more  about  the  Anasazi. 

2.57 

.98 

13.  If  I had  the  opportunity,  I would  choose  to  learn  about  another  topic 
using  a computer  program  like  this  one. 

1.97 

.88 

14. 1 enjoyed  learning  the  material  in  the  “Who  Were  the  Anasazi” 
and  in  the  “Where  Did  They  Live”  sections. 

2.05 

.66 

15.  The  material  in  the  “Who  Were  the  Anasazi”  and  in  the 
“Where  Did  They  Live”  sections  was  presented  effectively. 

1.84 

.53 

16. 1 enjoyed  learning  the  material  in  the  “What  Happened  to  the 
Anasazi”  section. 

2.17 

.82 

17.  The  material  in  the  “What  Happened  to  the  Anasazi” 
section  was  presented  effectively. 

1.88 

.69 

18. 1 enjoyed  learning  the  material  in  the  “How  Did  the  Anasazi  Live?” 
section. 

1.92 

.74 

19.  The  material  in  the  “How  Did  the  Anasazi  Live?”  section  was 
presented  effectively. 

1.81 

.66 

20. 1 enjoyed  learning  the  material  in  the  “What  Did  They  Use  to 
Make  a Living?”  section. 

2.05 

.75 

21.  The  material  in  the  “What  Did  They  Use  to  Make  a Living?” 
section  was  presented  effectively. 

1.86 

.66 

Note:  N = 1 10.  Scores  ranged  from  1 (Strongly  Agree)  to  5 (Strongly  Disagree). 


Navigation  and  Choice  Patterns 

Mean  scores  on  time  and  learner  navigation  and  choice  patterns  are  shown  in  Table  4.  Students  chose  to  return 
to  instruction  from  practice  exercises  an  average  of  2.8  times,  and  chose  previous  and  back  options  an  average  of  12.8 
times.  The  average  total  amount  of  time  spent  by  students  on  instruction  and  practice  was  about  28  minutes  (1679 
seconds).  The  mean  number  of  instruction  and  practice  cards  viewed  was  193  per  student.  As  the  total  program  included 
133  cards,  including  instruction,  direction,  questionnaire  and  test  screens,  this  number  indicates  that  many  students 
viewed  instruction  and  practice  cards  more  than  once. 
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Table  4 

Mean  Scores  on  Learner  Time  and  Navigation  Variables 


Variable Mean SD 

Number  of  Times  Chose  to  Return 
to  Instruction  from  Practice 

INSTRTRN  2.8  2.7 

Number  of  Times  Chose 
Previous  and  Back 

PRRTCNT  12.8  19.6 


Time  (in  seconds)  Spent  on 
Instruction  and  Practice 

TOTTIME  1679  813 

Total  Number  of  Instruction  and 
Practice  Cards  Viewed 

TOTCOUNT  193  112 


Multiple  regression  analysis  performed  to  investigate  the  predictive  value  of  several  student  time  and  navigation 
patterns  on  posttest  scores  is  shown  in  Table  5.  This  analysis  revealed  that  both  total  time  spent  on  instruction  and 
practice  cards  (Tottime)  and  number  of  times  previous  or  back  selections  were  made  (Prrtcnt)  were  significant  predictors 
of  achievement  (p<.05). 


Table  5 

Backward  Regression  Analysis  on  Choice  and  Navigation  for  Posttest  Results 


Variable  Remaining 

R Square 

F 

P 

prrtcnt  & tottime 

.06190 

3.530 

.0328 

prrtcnt 

.04489 

5.07609 

.0263 

* Significant  @ the  .05  level 


Students*  Perceptions  of  How  They  Used  the  Program 

Data  gathered  from  the  “notebook”  feature  is  shown  in  Table  6.  Ninety-eight  of  the  110  students  used  the 
notebook  feature.  The  most  frequently-cited  response  described  what  they  were  doing  in  the  program  (86),  such  as 
answering  questions  or  taking  quizzes.  Almost  as  many  students  (85)  indicated  what  they  liked  about  the  program, 
while  33  described  features  they  did  not  like  about  it.  Many  students  (84)  stated  that  they  were  learning  about  Anasazi 
Indians,  while  33  students  said  that  the  program  was  difficult.  Thirty  students  volunteered  that  they  liked  having  the 
option  to  go  back  and  review  screens  and  information  and  28  students  stated  that  they  were  using  different  mouse  tools. 
Twelve  students  wrote  that  they  had  no  comment. 
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Table  6 

Students'  Perceptions  of  How  They  Were  Using  the  Anasazi Computer  Program 


Resoonses 

Number  of  Responses 

Answering  practice  questions,  taking  practice 
quizzes,  taking  final  quiz,  reading  text,  doing 
surveys,  taking  notes. 

86 

Liked  program,  program  is  interesting,  easy  to  use, 
information  is  clear,  likes  immediate  feedback,  likes 
pictures  and  sound. 

85 

Learning  about  Anasazi  Indians,  going  through 
program. 

84 

Program  is  hard  and  confusing,  got  lost  in  program, 
program  is  long  and  boring,  don’t  like  program. 

33 

Would  rather  learn  from  person  or  book,  problems 
using  mouse,  survey  too  long,  notebook  feature  is 
bothersome. 

33 

Likes  being  able  to  go  back  and  review  information  in 
program,  using  review  options. 

30 

Using  mouse,  buttons,  arrows,  etc.. 

28 

No  comment. 

12 

Note:  N - 98  out  of  1 10  subjects  used  the  notebook  feature.  Subjects  could  mention  a number  of  responses.  Categories 

were  collapsed. 


Discussion 

This  study  is  one  of  the  first  to  investigate  learners'  patterns  of  choices  and  navigation  pathways  in  a completely 
free-access  network-structure  hypermedia  program.  There  were  no  effects  in  this  study  for  incentive  treatment  on  either 
performance  or  attitudes.  This  is  likely  to  have  been  a result  of  the  compressed  high  scores  on  the  posttest.  Most 
students  did  very  well,  with  a mean  of  17  out  of  20.  This  brings  to  mind  Clark's  (1983)  contention  that  well-designed 
instructional  methods  are  the  key  to  success.  Both  treatments  included  carefully-designed  instruction,  which  matched  the 
instructional  objectives  and  test  items  to  the  lesson  content.  Students’  responses  to  attitude  items  also  indicated  their 
satisfaction  with  the  instructional  design  of  the  CAI  program. 

The  results  of  this  study  also  indicate  that  college  students  generally  like  CAI  programs,  even  those  which  teach 
content  they  may  initially  have  been  neutral  about.  Contrary  to  expectations,  it  appears  that  their  prior  interest  may  not 
be  related  to  their  performance.  Similarly,  their  attitudes  toward  the  CAI  program  were  generally  favorable.  These 
attitudes,  however,  were  also  not  related  to  their  performance. 

Results  of  this  study  did  indicate  that  there  may  be  a relationship  between  performance  and  prior  experience,  not 
with  computers,  but  specifically  with  routine  integration  of  CAI  into  college  courses  to  teach  content.  The  results  of 
this  study  indicate  that  the  more  experience  students  have  with  CAI  the  better  this  method  may  enhance  their  learning  of 
content. 
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A similar  relationship  between  performance  scores  and  students'  desire  to  learn  other  content  using  CAI  may 
indicate  that  more  able  students  like  learning  using  computers  better.  This  result,  too,  seems  to  support  the  idea  of 
providing  college  students  with  more  experience  using  computers  to  learn,  if  that  is  to  be  the  method  of  delivery  in  a 
course. 

Other  results  indicate  that  the  total  time  spent  on  instruction  and  practice  cards  and  the  number  of  previous  or 
back  selections  made  were  significant  predictors  of  achievement.  That  time  spent  on  instruction  affected  achievement  was 
not  surprising.  The  effects  of  students'  decisions  to  go  back  to  previous  instruction,  however  may  be  an  important 
finding  for  developers  of  CAI.  Supporting  data,  in  the  form  of  students’  perceptions  of  how  they  used  the  program,  also 
indicate  that  students  liked  having  the  option  to  go  back  to  previous  screens  wherever  and  whenever  they  desired. 

Results  of  several  previous  learner-control  studies  indicate  that  learners  do  not  know  how  to  manage  their  own 
learning  to  improve  achievement  (Pollack  and  Sullivan,  1990;  Ross  and  Rakow,  1981;  Hannafin  & Sullivan,  1994). 
However,  in  these  studies  students  were  either  required  to  make  choices  before  the  instruction  began  or  as  they  were 
learning  new  material.  Students  were  not  given  the  option  to  manage  their  own  review  of  previously  viewed  screens  of 
information.  The  findings  from  the  present  study  indicate  that  even  though  learners  may  not  be  able  to  make 
instructional  choices  that  improve  achievement  prior  to  learning  new  material,  collegiate  learners  may  be  able  to  judge  in 
what  fashion  they  need  to  review  previously-viewed  instruction  and  practice  in  order  to  achieve  higher  posttest  scores. 

The  results  of  one  study  cannot  be  taken  as  a prescription  for  designers  or  researchers.  It  is  suggested  that  future 
studies  be  conducted  investigating  the  effects  of  going  back  and  other  patterns  of  learners’  navigational  behaviors  in  free- 
access  CAI  programs  more  deeply.  Studies  should  also  be  conducted  with  other  types  of  learners  in  other  settings,  to 
determine  the  learning  and  attituainal  effects  of  incentive  treatments,  and  navigational  patterns  and  choices,  as  well  as  the 
relationships  among  prior  interest,  prior  experience,  attitudes  and  performance. 


Table  7 

Post-instructional  Attitude  Data  by  Treatment 


Treatment 

Mean 

SD 

N 

Performance  incentive 

40.60 

10.01 

55 

Task  incentive 

40.42 

8.91 

55 

Total 

40.51 

9.43 

110 
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The  average  person  can  distinguish  between  10,000  and  20,000  colors  (Pettersson,  1993;  Tufte,  1990).  State-of- 
the-art  computers  being  used  today  for  instructional  multimedia  products  have  monitors  capable  of  displaying  different 
colors  beyond  these  figures  and  well  into  the  millions.  Designers  are  given  virtually  unlimited  control  over  the  factors 
that  govern  the  display  of  those  colors.  Not  only  can  designers  decide  when  and  where  to  use  color,  but  they  can  also 
determine  the  physical  characteristics  of  the  colors  themselves.  According  to  Keller  and  Burkman  (1993),  people  have 
come  to  expect  media  products  to  have  high-quality  color,  and  indeed,  most  prefer  instructional  products  that  have  color 
over  those  that  do  not.  With  the  ready  availability  of  color  on  today’s  computers,  what  is  a designer  supposed  to  do? 
There  are  so  many  colors,  so  many  choices. 

This  paper  first  briefly  examines  how  color  is  displayed  on  modem  computer  monitors  and  how  humans 
perceive  color.  The  three  following  sections  consider  three  primary  uses  for  color:  affective,  structural,  and  cognitive.  The 
succeeding  section  discusses  problems  inherent  to  the  use  of  color  and  offers  some  solutions.  Finally,  the  paper  concludes 
with  recommendations  for  the  proper  and  effective  use  of  color  in  instructional  multimedia  products. 

What  Is  Color,  and  How  Does  It  Get  on  a Computer  Monitor? 

Color  is  a sensation  and  not  a physical  entity,  according  to  Humphreys  (1993).  Murch  (1983)  maintained  that 
the  way  we  perceive  color  is  the  result  of  many  factors:  the  capacity  of  the  object  in  question  to  reflect  and  absorb  certain 
wavelengths  of  light;  the  properties  of  the  light  source  illuminating  the  object;  the  medium  through  which  the  light 
travels;  the  properties  of  the  surrounding  objects;  the  biochemical  state  of  the  eyes  of  the  observer;  and  the  observer’s 
previous  experience  with  the  object  or  its  colors.  Although  all  of  these  factors  are  of  concern  to  the  designer,  he  or  she 
usually  has  control  over  the  first  factor  alone.  That  is,  the  designer  can  only  specify  what  colors  the  monitor  should 
display.  Because  of  the  intervening  factors  between  the  display  and  the  eyes  of  the  observer,  the  designer  cannot  control 
what  colors  the  observer  actually  perceives. 

As  Norman  (1990)  pointed  out,  although  color  organization  can  be  traced  back  to  the  ancient  Greeks,  it  was  Sir 
Isaac  Newton  who  first  brought  us  the  now-familiar  color  wheel,  the  principles  of  which  have  become  an  underlying 
assumption  for  virtually  every  color  model  since.  An  example  of  one  of  these  principles  is  that  mixing  together  red, 
green,  and  blue  paint  produces  a mixture  approaching  black.  Producing  new  colors  in  this  manner  is  referred  to  as  an 
additive  process  and  is  the  one  with  which  most  people  are  familiar.  Unfortunately,  color  on  a computer  monitor  is  not 
accomplished  by  mixing  colored  pigment,  but  rather  by  mixing  colored  light.  In  this  system,  called  subtractive, 
combining  red,  green,  and  blue  light  produces  white.  According  to  Norman,  prior  to  the  advent  of  color  computers,  very 
few  people  (theatre  lighting  designers  being  one  exception)  had  the  opportunity  to  combine  colors  using  a subtractive 
process. 

More  specifically,  a monitor’s  cathode  ray  tube  produces  an  image  when  electrons  strike  the  phosphorescent 
material  lining  the  front  interior  of  the  glass.  As  Thorell  and  Smith  (1990)  described,  the  phosphors  are  arranged  in  triads 
of  three  primary  colors:  red,  blue,  and  green.  These  phosphors  are  individually  energized  by  three  different  electron  beams 
and  hence  selectively  lighted.  Illumination  of  all  three  phosphors  in  a triad  produces  white,  and  illuminating  none 
produces  black.  Stimulating  different  combinations  in  the  triad  produces  secondary  colors  (for  example,  lighting  the  red 
and  green  phosphors  results  in  yellow).  By  varying  the  intensity  of  the  electron  beams,  a full  spectrum  of  colors  can  thus 
be  produced. 

While  color  on  a computer  is  constructed  using  red,  green,  and  blue  phosphors,  humans  commonly  refer  to  color 
by  three  different  interrelated  components:  hue,  saturation,  and  lightness.  Murch  (1985)  defined  hue  as  the  basic 
component  of  color  (which  depends  on  the  specific  wavelengths  of  light  received),  saturation  as  the  purity  of  hue  (which 
depends  on  the  range  of  wavelengths  received),  and  lightness  as  the  amount  of  light  reflected  (which  depends  on  the 
presence  and  amount  of  achromatic  color — black,  gray,  and  white).  For  example,  “red”  is  a word  used  to  describe  hue; 
“pink”  is  a v ord  used  to  describe  saturation;  and  “dark”  is  a word  used  to  describe  lightness.  So  when  someone  describes 
an  object  as  being  “dark  pink,”  for  example,  he  or  she  has  simultaneously  used  hue,  saturation,  and  lightness  in  the 
description. 

San  it  (1983)  contended  that  it  is  important  for  the  designer  to  understand  how  a computer  display’s  system 
works  since  c olors  on  a computer  are  produced  in  the  unfamiliar  additive  red-green-blue  (RGB)  system,,  but  are  perceived 
in  the  familiar  hue-lightness-saturation  (HLS)  system.  Murch  (1983)  described  how  the  gap  between  these  two  color 
systems  is  bridged  by  most  computer  systems  having  an  interface  which  allows  the  desif  ler  to  manipulate  controls  for 
hue,  lightness,  and  saturation  directly  and  interactively. 
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The  Affective  Role  of  Color 

Many  authors  have  argued  that  color  adds  a strong  affective  dimension  to  visual  displays.  Pettersson  (1993) 
suggested  that  color  enhances  the  perception  of  a visual  message;  Shneiderman  (1987)  concluded  that  color  can  add 
accents  to  an  uninteresting  display;  Tufte  (1990)  maintained  that  color  enlivens  and  informs  the  computer  user;  Dwyer 
and  Lamberski  (1983)  noted  that  color  makes  displays  attractive  and  emotionally  appealing;  and  Dondis  (1973)  claimed 
that  color  is  “the  aesthetic  frosting  on  the  cake”  (p.  50).  More  substantively,  Samit  (1983)  reported  that  tests  have 
indicated  that  viewers  actually  appear  to  feel  they  have  a better  understanding  when  images  are  displayed  in  color. 

As  Davidoff  (1991)  observed,  there  are  particularly  strongly  held  beliefs  about  emotional  responses  to  color.  For 
example  the  color  red  is  said  to  be  make  one  feel  warmer,  to  arouse  one  to  action,  to  accelerate  the  passage  of  time,  and 
to  increase  one’s  strength.  On  the  other  hand,  Sharpe  (1974)  noted  that  blue  (at  the  opposite  end  of  the  spectrum  from 
red)  is  said  to  make  one  feel  cooler,  calmer,  and  more  in  control.  Davidoff  reported  that  there  are  very  few  well- 
documented  and  controlled  studies  that  actually  verify  the  effects  of  color  on  behavior.  What  is  important  for  our  purposes 
is  that  people  believe  that  color  affects  them  both  emotionally  and  physically. 


Structural  Uses  of  Color 

When  designing  screens  for  the  computer-user  interface,  the  designer  must  consider  how  the  structure  and  the 
various  functions  of  the  interface  will  be  communicated  to  the  user  (Brown,  1989;  Faiola  & DeBloois,  1988;  Laurel, 

1990;  Pettersson,  1989;  Shneiderman,  1987).  Appropriate  use  of  color  can  be  used  as  an  element  in  this  communication 

process.  ^ ^ way  which  color  can  be  used  structurally  is  in  screen  menu  design.  As  an  example,  Shneiderman 
(1987)  suggested  showing  all  menu  items  in  one  color,  the  title  in  another  color,  the  instructions  in  a third  color,  and 
error  messages  in  yet  a fourth  color.  When  screen  space  is  at  a premium,  he  also  recommended  using  dissimilar  colors  to 
distinguish  close  but  logically  distinct  fields.  Rambally  and  Rambally  (1987)  also  suggested  using  different  colors  to 

separate  prompts,  commands,  input/output  (I/O)  fields,  and  the  like. 

Using  color  for  text  is  another  option  available  to  the  designer.  Both  foreground  and  background  colors  need  to 
be  taken  into  account.  Isaacs  (1987)  and  Brown  (1989)  both  advised  that  the  two  colors  should  be  chosen  so  that  the 
greatest  contrast  in  lightness  is  obtained.  Keyes  (1993)  noted  that  since  black  and  white  have  the  maximum  contrast  in 
lightness  this  combination  has  maximum  visibility.  Keyes  added  that  instead  of  black,  dark  shades  of  green,  blue,  and 
violet  could  be  used  with  only  a small  decrease  in  legibility.  Assuming  that  the  lightness/contrast  rule  is  followed. 
Clausing  and  Schmitt  (1989, 1990)  demonstrated  that  the  colors  chosen  have  no  significant  effect  on  either  reading  rate  or 
comprehension.  Nevertheless,  numerous  authors  have  offered  specific  suggestions  for  which  colors  to  use.  Humphreys 
(1993)  maintained  that  dark  letters  on  a light  background  were  more  legible  than  the  reverse,  Pettersson  (1989)  proffered 
black  or  dark  brown  text  on  a light  yellow  background  as  being  the  most  legible,  Gillingham  (1988)  cited  a Swedish 
study  that  c laimed  blue  or  green  on  white  led  to  increased  accuracy  and  higher  subjective  ratings,  and  Isaacs  concluded  that 
green  text  on  a black  background  is  the  best  choice. 

Colored  text  can  also  be  used  to  provide  signals  to  the  user.  Marchionini  (1988)  suggested  that  color  could  be 
used  in  hypertext  to  indicate  .inks.  Cook  and  Kazlauskas  (1993)  recommended  using  two  different  colors  in  feedback  after 
an  incorrect  response:  one  color  to  indicate  what  the  user  should  do  next  and  a different  color  to  elaborate  on  why  the  user 
was  incorrect.  Keyes  (1993)  advocated  using  color  for  information  types  that  are  difficult  to  signal  typographically,  such 
as  warnings,  hints,  user-entered  information,  cross-references,  and  section  dividers. 

Shneiderman  (1987)  suggested  that  when  exceptional  conditions  or  time-dependent  information  must  be  relayed, 
color  can  be  used  to  attract  the  user's  attention.  Occasionally,  there  are  times  when  the  computer  requires  the  user  to 
attend  to  something  unrelated  to  the  user’s  current  task.  If  the  computer’s  need  is  not  urgent,  rather  than  having  it 
forcibly  and  abruptly  interrupt  the  user  with  a dialog  box,  Baecker  and  Small  (1990)  proposed  having  a less  intrusive 
alerting  icon  that  changes  color  to  become  progressively  more  red  as  it  is  ignored  for  an  increasingly  longer  period  of 
time.  If  time  is  more  of  the  essence,  Faiola  and  DeBloois  ( 1 988)  advocated  using  highly  saturated  colors  to  attain  a quick 

response. 

When  designing  the  structure  of  an  interface  with  color,  designers  should  also  be  aware  that  color  can  provide 
some  overall  physiological  benefits.  According  to  Tufte  (1990),  softening  a bright  white  background  with  color  calms 
video  glare.  Smith  (1987)  reported  that  the  use  of  multicolor  screens  have  reduced  reports  of  visual  stress  or  fatigue  from 
users  who  had  previously  viewed  information  on  black-and-white  or  single-color  screens.  Smith  also  contended  that 
information  on  a multicolor  screen  can  be  perceived  at  a greater  distance  than  a black-and-white  version  of  the  same 
information. 
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Cognitive  Uses  of  Color 

Color  may  also  be  useful  for  enhancing  learning.  Faiola  and  DeBloois  (1988)  claimed  that  color  “can  aid 
memory  and  enhance  the  understanding  of  information”  (p.  16),  Dwyer  and  Lamberski  (1983)  postulated  that  color  can 
make  instructional  products  “more  effective  in  facilitating  student  achievement  of  specific  kinds  of  learning  objectives” 

(p.  304),  Milheim  and  Lavix  (1992)  declared  that  “color  can  be  used  effectively  ...  to  aid  in  student  learning”  (p.  18), 
and  Keyes  (1993)  averred  that  color  can  create  “a  visual  layer  that  we  separate  perceptually”  from  other  information  (p. 
646).  Beyond  the  generalities,  in  what  specific  ways  can  color  enhance  learning?  Numerous  researchers  and  authors  have 
suggested  using  color  to  highlight  salient  features,  color  code  related  bits  of  information,  decrease  the  cognitive  load,  and 
simplify  complex  information. 

Highlighting  salient  features . As  Tufte  (1990)  noted,  foremost  among  color’s  attributes  is  its  ability  to 
make  things  stand  out,  to  draw  the  user’s  attention.  Borrowing  from  classical  cartography,  Tufte  noted  that  small  spots  of 
intense,  saturated  color  can  be  used  effectively  to  convey  information  by  making  it  stand  out  from  the  rest  of  the 
illustration.  Winn  (1993)  added  that  the  highlighting  color  doesn’t  even  have  to  correspond  to  the  colors  of  things  in  the 
real  world:  using  bright  red  to  color  the  forearm  of  an  athlete  throwing  a javelin,  for  example,  is  not  meant  to  imply  that 
the  arm  is  in  fact  red,  but  rather  to  draw  attention  to  the  position  of  the  arm.  In  a meta-analysis  of  the  research  on  the  use 
of  color  in  visual  displays,  Christ  (1975)  concluded  that  using  color  to  help  users  find  and  identify  features  is  as  much  as 
200%  more  effective  than  using  size,  brightness,  or  shape. 

Color  coding . Perhaps  no  instructional  use  of  color  has  been  as  thoroughly  considered  and  studied  as  color 
coding.  Color  coding  is  used  constantly  in  everyday  life,  allowing  us  to  readily  associate  particular  messages  with  certain 
colors  (Brown,  1989;  Durrett  & Trezona,  1982;  Faiola  & DeBloois,  1988;  Rambally  & Rambally,  1987).  According  to 
Durrett  and  Stimmel  (1987),  color  affects  the  coding  of  information  in  human  memory.  They  cited  numerous  studies  in 
which  color  was  used  as  v\  organizational  and  differentiating  factor  to  help  subjects  recall  and  retrieve  information. 
Somewhat  surprisingly,  Dwyer  and  Lamberski  (1983)  concluded  that  even  if  the  colors  chosen  do  not  contribute  to  the 
message  content,  color  can  nevertheless  still  facilitate  the  retrieval  of  essential  learning  cues. 

Christ  (1984)  offered  one  of  the  simplest  ways  to  use  color  coding:  identifying  categorical  information.  The 
number  of  categories  may  be  as  few  as  two,  such  as  in  air  traffic  controller  displays  where  one  color  is  used  to  designate 
planes  over  a certain  altitude  and  another  color  for  planes  below.  Christ  noted  that  although  with  training  users  can  learn 
to  increase  the  i jmber  of  identifiable  colors  in  a display,  he  recommended  using  fewer  than  ten.  Others  have  advocated  as 
few  as  three  (Milheim  & Lavix,  1992)  to  as  many  as  eleven  (Rambally  & Rambally,  1987).  Other  applications  of  using 
color  to  categorize  information  include  parsing  the  different  parts  of  mathematical  statements  (Thorell  & Smith,  1990), 
showing  the  nesting  levels  of  a block-structured  programming  language  (Shneiderman,  1987),  and  delineating  the 
contours  in  an  architectural  drawing  (Norman,  1990). 

Color  can  also  be  used  to  order  logically  related  data.  In  an  example  from  chemistry,  Thorell  and  Smith  (1990) 
suggested  using  different  saturations  of  the  same  color  to  code  different  concentration  levels — high  saturation  to  represent 
maximum  concentration  and  desaturation  for  minimum  levels.  Similarly,  in  organizational  charts,  executive  officers 
could  be  shown  in  saturated  colors  and  divisions  that  report  to  them  in  desaturated  values  of  the  same  colors.  In  an 
accounting  environment,  bright  red  could  be  used  to  indicate  a payment  extremely  overdue  and  lighter  shades  to  represent 
payments  overdue  by  fewer  days. 

Color  can  also  be  used  to  code  processes.  Among  their  numerous  examples,  Thorell  and  Smith  (1990) 
demonstrated  using  color  coding  to  indicate  the  decomposition  of  a heated  chemical  compound,  to  represent  the  body’s 
reactions  to  different  amino  acids,  to  signify  the  changes  in  stress  loadings  on  solid  objects,  and  to  assist  doctors  in 
interpreting  CAT  scans. 

Decreasing  the  cognitive  load L Keyes  (1993)  postulated  that  we  can  use  color  to  extend  the  cognitive 
limit  by  creating  a visual  layer  separate  from  monochromatic  typographic  and  spatial  cues.  Because  color  is  perceived 
preattentively  (automatically),  less  effort  is  required  and  the  user  can  handle  more  information.  In  what  is  known  as  the 
neurophysiology  of  modularity,  Davidoff  (1991)  asserted  that  in  the  visual  system,  color  provides  a secondary,  pathway 
to  the  brain,  parallel  to  that  for  other  visual  information.  Adding  credence  to  the  hypothesis  of  modularity,  Durrett  and 
Stimmel  (1987)  related  an  experiment  in  which  color  was  shown  to  be  an  extremely  effective  adjunct  in  learning 
nonsense  syllables. 

Cognitive  load  is  closely  allied  to  color  coding.  For  example,  since  red  is  traditionally  associated  with  stop, 
yellow  with  caution,  and  green  with  go  (Brown,  1989;  Durrett  & Trezona,  1982;  Faiola  & DeBloois.  1988; 
Shneiderman,  1987),  users  do  not  have  to  learn  new  associations  between  these  colors  and  their  meanings.  Thorell  and 
Smith  (1990)  added  that  to  most  people,  light  colors  give  the  impression  of  large  sizes,  light  weights,  tall  heights,  and 
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close  distances;  dark  colors  give  the  opposite  appearances.  Accordingly,  designers  should  be  able  to  incorporate  these 
associations  with  colors  into  their  designs  and  expect  users  to  respond  accordingly. 

Simplifying  complex  information.  Color’s  propensity  to  make  things  stand  out  combined  with  its 
ability  to  code  related  items  makes  it  an  effective  way  to  simplify  complex  information.  Keyes  (1993)  argued  that  color 
simplifies  by  visually  organizing  and  classifying  information,  clarifying  both  differences  and  relationships.  As  a simple 
example*,  Thorell  and  Smith  (1990)  described  using  colors  to  show  the  magnitude  of  change  in  an  event:  small  changes  in 
color  (blue  to  green,  for  example)  to  signify  small  changes  in  magnitude  and  large  changes  in  color  (blue  to  orange)  for 
large  changes. 

In  addition  to  perspective,  artists  have  long  used  color  in  three-dimensional  drawings  to  separate  the  near  from 
the  far.  Norman  (1990)  noted  that  grayer,  desaturated  colors  appear  farther  away  than  more  intense  colors,  giving  the 
illusion  of  depth.  Helping  a user  visualize  three  dimensions  on  the  flatland  of  the  screen  can  have  profound  learning 
effects.  For  example,  Mayes,  Kibby,  and  Anderson  (1989)  observed  that  significant  advances  were  made  in  the 
understanding  of  molecular  chemistry  with  the  advent  of  computer  graphic  systems  capable  of  displaying  complex  three- 
dimensional  molecular  structures  using  color  to  show  grouping. 

Problems  and  Solutions 

As  with  any  design  element,  color  has  certain  potential  problems  associated  with  its  use.  Many  authors  have 
alerted  us  to  the  “color  blindness”  of  some  users  (Brov'n,  1989;  Durrett  & Stimmel,  1987;  Humphreys,  1993;  Rambally 
& Rambally,  1987;  Shneiderman,  1987;  Thorell  & Smith,  1990).  As  Thorell  and  Smith  explained,  our  eyes  have 
receptors  called  cones  which  fall  into  three  groups  according  to  their  sensitivity  to  the  primary  colors  of  red,  green,  and 
blue.  Deficient  color  vision  occurs  when  any  one  or  more  of  these  three  groups  has  insufficient  sensitivity.  True  color 
blindness — being  able  to  see  only  shades  of  gray — is  exceedingly  rare.  Silverstein  (1987)  put  the  figure  at  less  than 
0.003%  (3  people  in  100,000).  Partial  color  blindness  is  the  total  inability  to  perceive  one  of  the  three  primary  colors. 
Silverstein  estimated  that  a little  over  2%  of  the  population  have  one  or  more  forms  of  this  deficiency.  The  most 
common  color  deficiency  is  not  really  color  blindness,  but  rather  color  weakness.  According  to  Silverstein, 
approximately  1%  of  the  population  is  weak  in  perception  of  red  and  another  5%  in  perception  of  green.  Consequently, 
about  8%  of  the  population  has  some  sort  of  color  deficiency.  Depending  on  the  type,  color  deficiencies  are  10  to  100 
times  more  prevalent  in  males  than  in  females.  It  is  important  to  note  that  all  color  deficiencies  affect  only  the  perception 
of  different  hues — all  color-deficient  individuals  can  still  perceive  differences  in  lightness  and  saturation. 

There  are  several  ways  in  which  the  designer  can  compensate  for  color  deficiencies.  As  has  already  been  noted  in 
several  examples,  instead  of  using  different  hues,  the  designer  can  use  either  different  saturations  or  different  levels  of 
lightness  to  represent  different  entities.  When  different  hues  are  deemed  necessary,  however,  Keyes  (1993)  recommended 
varying  the  individual  saturations  so  that  the  contrast  is  accentuated  (for  example,  instead  of  using  bright  red  and  bright 
green,  consider  using  bright  red  and  pale  green — even  the  totally  color  blind  will  be  able  to  perceive  the  difference). 
Another  alternative  is  to  use  what  is  called  “redundant  cueing.”  Instead  of  using  only  color  to  signify  differences,  Durrett 
and  Trezona  (1982),  for  example,  suggested  using  shapes  or  labels  in  addition  to  colors. 

Because  of  the  physiological  structure  of  our  eyes,  certain  colors  create  certain  problems.  One  limitation 
outlined  by  Humphreys  (1993)  is  that  the  lens  of  the  eye  is  not  color  corrected,  thereby  causing  pure,  saturated  colors  at 
the  same  distance  to  appear  to  be  at  different  distances.  Along  these  same  lines,  Samit  (1983)  explained  how  the  eye 
cannot  precisely  focus  on  all  colors  together.  For  instance,  since  the  eye  cannot  focus  on  red  and  blue  simultaneously,  the 
muscles  will  be  forced  to  focus  alternately  on  one  color  and  then  by  the  other,  with  resulting  eyestrain.  Samit  also  noted 
that  because  the  center  of  the  retina  is  nearly  devoid  of  blue  receptors  (even  in  people  with  normal  color  vision),  small 
blue  objects  virtually  disappear  when  we  try  to  focus  on  them.  Two  solutions  present  themselves  immediately:  avoid 
pure,  saturated  colors  at  opposite  ends  of  the  spectrum  (Durrett  & Trezona,  1982),  and  don’t  use  blue  for  fine  lines  and 
text  (Humphreys). 

Another  problem  with  using  color  is  that  not  everyone  responds  to  color  in  the  same  way.  There  are  cultural, 
gender,  age,  and  occupational  differences.  For  example,  Thorell  and  Smith  (1990)  pointed  out  that  in  Japan  green  implies 
youth  and  energy,  but  in  France  it  connotes  criminality.  In  the  United  States,  yellow  stands  for  caution  and  cowardice, 
while  it  signifies  happiness  and  prosperity  in  Egypt.  Several  studies  have  attempted  to  show  that  males  and  females  differ 
in  their  color  preferences  and  responses  (Davidoff,  1991;  Krishna,  1972;  Sharpe,  1974).  Sharpe  claimed  that  older  people 
prefer  bright  primary  colors  (perhaps  because  color  acuity  declines  with  age),  while  Durrett  and  Stimmel  (1987)  found 
that  children  are  not  able  to  utilize  color  coding  as  quickly  as  adults  can.  While  blue  represents  corporate  reliability  to 
financial  managers  (as  in  “Big  Blue”  |IBM)),  Thorell  and  Smith  pointed  out  that  blue  represents  death  to  health  care 
professionals  (as  in  “code  blue”);  red  means  danger  to  process  control  engineers,  but  healthy  to  health  care  professionals. 


Again,  the  solution  is  for  the  designer  to  know  the  audience  and  to  conduct  prototyping  and  user-acceptance  tests  (Brown, 
1989). 

Using  colors  inappropriately  or  using  too  many  colors  can  create  what  Albers  called  “1  + 1 = 3 or  more”  (as 
cited  in  Tufte,  1990,  p.  53).  That  is,  certain  uses  and  combinations  of  colors  can  create  unintentional  artifacts  or  visual 
“noise”  (Tufte,  1989).  We  have  all  seen,  for  example,  the  kind  of  optical  illusion  where  the  image  seems  to  literally 
vibrate  because  of  the  intensity  of  the  colors.  Tufte’s  (1989)  solution  was  to  use  the  colors  found  in  nature,  since  they 
offer  some  measure  of  a accepted  visual  harmony. 


Recommendations 

Color  is  undeniably  a part  of  our  lives.  We  use  colors  to  help  identify,  categorize,  a-  j locate  things.  We  use 
color  terms  to  express  emotions  (“feeling  blue,”  “in  the  pink,”  “green  with  envy,”  and  “yellow-bellied”),  to  ascribe 
motives  (“yellow  journalism,”  “pinko,”  “red  herring,”  and  “true  blue”),  to  denote  financial  states  (“in  the  red”  and  “in  the 
black”),  to  describe  qualities  (“purple  prose,”  “greener  pastures,”  and  “red  letter  day”),  to  compare  things  (“more  [of 
something]  than  there  are  colors  in  the  rainbow,”  “black  and  white,”  and  “shades  of  gray”),  to  express  actions  (“color  my 
opinion”  and  “looking  at  the  world  through  rose-colored  glasses”),  and  to  advertise  (“Coca-Cola  red,”  “Pepto-Bismol 
pink,”  and  “Kodak  yellow”).  It  seems  only  appropriate  that  designers  should  take  advantage  of  what  color  has  to  offer  in 
both  the  affective  and  cognitive  domains. 

In  order  to  incorporate  color  aesthetically  and  effectively  into  instructional  multimedia  products,  designers  have 
at  their  disposal  a wealth  of  information  from  the  fields  of  graphic  design,  fine  arts,  psychology,  behavioral  science, 
physiology,  optics,  cognitive  science,  semiotics,  linguistics,  cartography,  human  factors  engineering,  ergonomics, 
military  science,  architecture,  and  anthropology.  Knowing  how  color  is  produced  and  perceived  should  help  the  designer 
better  understand  the  contributions  (and  detriments)  that  color  can  offer.  Although  good  instruction  can  be  accomplished 
without  using  any  color  whatsoever,  color  may  provide  an  additional  factor  in  human  encoding  processes. 

Because  computers  now  have  millions  of  colors  available,  there  is  a tendency  for  some  designers  to  overdo  it. 
The  best  advice  in  this  regard  came  from  Tufte  (1990):  “Above  all,  do  no  harm”  (p.  81).  Too  much  color  can  be 
distracting,  and  has  even  been  shown  to  degrade  performance  on  memory  and  recognition  tasks  (Christ,  1975,  1984; 
Dwyer  & Lamberski,  1983).  Screen  designers  would  do  well  to  follow  the  lead  of  graphic  artists  who  design  in  black, 
white,  and  shades  of  gray  before  they  ever  add  color.  This  practice  enables  them  to  judge  overall  balance,  harmony,  and 
clarity  (and  discover  any  potential  problems  for  the  color  deficient). 

The  full  range  of  uses  for  color  has  yet  to  be  fully  explored.  Some  of  what  Salomon  (1990)  called  “new  uses  for 
color”  (p.  269)  hold  promise  for  instructional  products.  As  an  example,  Salomon  explained  how  a document’s  icon  on 
the  desktop  could  be  colored  to  indicate  its  age — yellow  for  new  documents,  through  progressively  duller  shades  of  brown 
for  older  documents.  Color  could  also  be  used  to  provide  additional  functional  information  to  the  user.  Most  computer 
operating  systems  today  use  some  sort  of  icon  to  indicate  that  the  computer  is  currently  busy — the  wristwatch  on  the 
Macintosh  and  the  hourglass  in  Windows  are  two  such  examples.  When  the  user  might  be  interested  in  knowing  more 
precisely  the  internal  workings  of  the  computer,  Baecker  and  Small  (1990)  recommended  having  an  icon  that  is  blue  to 
indicate  that  the  current  action  is  totally  CPU-bound,  yellow  to  indicate  totally  I/O-bound,  and  colors  in  between  blue  and 
yellow  to  indicate  more  balanced  processes.  Color  has  also  started  to  play  a role  in  the  implementation  of  so-called 
guides.  The  Macintosh’s  System  7.5,  for  example,  uses  color  dynamically  to  highlight  the  next  step  in  a process.  Color 
could  also  be  used  to  help  young  users  locate  hidden  or  secret  information  in  a game-like  environment.  By  having  the 
cursor  change  color  as  it  is  moved  about  the  screen,  children  could  be  told  when  they  are  “getting  hot”  (red)  or  when  they 
are  “getting  cold”  (blue). 

Much  has  been  written  about  the  instructional  use  of  color.  However,  authors  disagree  on  advice  and  their 
studies  often  contradict  one  another.  It  is  clear  that  additional  research  is  necessary.  Durrett  and  Stimmel  (1987)  noted  the 
following  irony: 

The  amount  of  research  done  on  color  as  a variable  in  learning,  attention,  and  coding  reached  a peak  in  the  late 
1960s  and  early  1970s,  declined  slightly  during  the  1970s,  and  declined  dramatically  between  the  late  1970s  and 
the  present.  This  is  paradoxical  because  the  use  of  color  in  the  media  of  instruction  and  the  potentials  for  use  of 
color  in  instruction  has  grown  dramatically,  (p.  249) 
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Classroom  teachers  have  long  used  group  activities  as  part  of  their  repertoire  of  teaching  techniques.  Having 
students  work  together  in  groups  sometimes  produces  meaningful  results  and  sometimes  doesn’t.  Personalities,  goals,  the 
group  dynamic,  the  teacher’s  level  of  supervision,  and  myriad  other  factors  all  play  a role  in  the  efficacy  of  the 
collaboration.  One  specific  kind  of  group  activity  which  has  a history  of  success  is  cooperative  learning.  Although  social 
psychological  research  on  cooperation  can  be  traced  back  to  the  1920s  (Slavin,  1977),  classroom  research  did  not  begin  in 
earnest  until  the  1970s.  At  that  time,  four  groups  of  researchers  working  independently  started  to  develop  and  research 
cooperative  learning  methods  for  the  classroom  (Slavin,  1990).  Slavin’s  team  at  Johns  Hopkins  University  developed 
“Student  Teams-Achievement  Divisions”  (STAD)  (Slavin,  1978)  and  ‘Teams-Games-Toumaments”  (TGT)  (DeVries  and 
Slavin,  1978);  Aronson  , Blaney,  Stephan,  Sikes,  and  Snapp  (1978)  designed  the  “Jigsaw”  technique;  Sharan  and  Sharan 
(1976)  produced  the  “Group  Investigation”  method  at  the  University  of  Tel- Aviv;  and  Johnson  and  Johnson  (1975) 
working  at  the  University  of  Minnesota  devised  the  “Learning  Together”  model  for  cooperative  learning.  Although  the 
precise  details  and  the  full  research  results  of  these  pioneering  efforts  are  beyond  the  scope  of  this  paper,  it  is  instructive 
for  the  purposes  of  this  paper  to  examine  the  basic  premise  of  each. 

Cooperative  Learning  Techniques 
Student  Teams-Achievement  Divisions 

In  STAD,  each  group  comprises  four  members,  selected  toward  achieving  a mixture  of  performance  levels, 
gender,  and  ethnicity.  The  teacher  presents  the  lesson  in  a conventional  manner,  after  which  the  students  work  within 
their  groups — now  called  “teams” — to  ensure  that  all  team  members  have  mastered  the  lesson.  Within  the  team,  the 
students  can  use  a variety  of  techniques  to  help  each  other  learn  the  material.  After  a suitable  period  of  time,  the  students 
then  take  a quiz  individually.  A student’s  quiz  score  is  then  compared  to  his  or  her  own  past  average,  and  points  are 
awarded  for  meeting  or  exceeding  that  average.  These  points  are  then  pooled  for  the  entire  team  to  arrive  at  a team  score. 
Teams  that  attain  a pre-specified  level  receive  certificates  or  other  rewards  (Slavin,  1986). 
Teams-Games-Toumaments 

Like  STAD,  TGT  has  the  same  teacher  presentation  of  the  lesson  and  the  same  group  composition.  Instead  of 
quizzes,  however,  a TGT-based  lesson  concludes  with  a tournament  in  which  students  compete  with  members  of  other 
teams  to  earn  points  for  their  own  teams.  A round  of  the  tournament  pits  students  against  other  students  with  similar 
past  achievement  records.  In  this  manner,  low  achievers  are  competing  against  other  low  achievers,  for  example.  As  a 
student’s  cumulative  achievement  record  changes,  he  or  she  is  “bumped”  to  a different  level.  As  during  the  quizzes  in 
STAD,  team  members  are  own  their  own  during  the  actual  tournament.  The  winner  of  the  round  contributes  points  to  his 
or  her  team’s  overall  score.  Also  as  with  STAD,  high-scoring  teams  earn  rewards  (Slavin,  1986). 

Jigsaw 

In  the  originally  Jigsaw  technique,  students  are  assigned  to  six-member  teams  to  work  on  academic  material 
which  has  been  divided  into  six  different  sections.  Each  team  member  then  studies  his  or  her  assigned  section.  Next,  all 
the  students  who  have  studied  the  same  section  (one  student  from  each  group)  meet  to  form  an  “expert  group”  to  discuss 
what  they  have  learned.  The  students  then  return  to  their  original  teams  and  take  turns  teaching  each  other  the  various 
sections  (Aronson,  et  al.,  1978). 

Group  Investigation 

Unlike  the  previous  methods,  the  groups  in  Group  Investigation  are  not  necessarily  assigned  by  the  teacher. 

Like  Jigsaw,  Group  Investigation  is  a task  specialization  method.  The  teacher  begins  by  assigning  a broad  topic  for 
inquiry.  The  students  then  break  the  topic  down  into  subtopics  based  on  their  prior  knowledge,  backgrounds,  and 
interests.  Teams,  ranging  in  size  from  two  to  six  students,  are  formed  by  those  sharing  a common  interest.  The  students 
then  investigate  the  topic  for  their  groups  and  prepare  a final  report  to  present  to  the  entire  class.  Traditionally,  evaluation 
for  Group  Investigation  focuses  on  higher-level  thinking  skills,  with  the  teacher  constantly  monitoring  each  student’s 
academic  activity  (Sharan  & Sharan,  1976). 

Learning  Together 

As  in  STAD  and  TGT,  learning  Together  is  based  on  four-to-five-member  groups  assigned  by  the  teacher.  The 
members  of  each  group  then  work  together  to  achieve  a common  goal,  usually  the  completion  of  an  assignment  sheet 
handed  out  by  the  teacher.  Individual  students  are  required  to  demonstrate  that  they  have  individually  mastered  the 
material.  Additionally,  Learning  Together  emphasizes  interpersonal  and  small-group  skills  by  having  the  students  discuss 
how  well  their  groups  are  working  together  to  achieve  the  goal.  The  developers  of  Learning  Together  recommended  team 
grades  rather  than  rewards  or  certificates  (Johnson  & Johnson,  1975). 


Student  Team  Learning 

Since  the  1970s,  these  same  researchers  and  others  have  continued  to  refine  the  older  cooperative  learning 
techniques  and  develop  new  ones.  According  to  Slavin  (1990),  more  than  half  of  all  studies  of  practical  cooperative 
learning  techniques,  including  those  cited  above,  involve  what  Slavin  called  “Student  Team  Learning”  methods.  The 
overarching  philosophy  behind  Student  Team  Learning  is  that  “the  students’  tasks  are  not  to  do  something  as  a team  but 
to  learn  something  as  a team”  (Slavin,  1990).  This  result  can  be  achieved  only  if  all  of  the  team  members  learn  the 
objectives  being  taught.  Regardless  of  the  name  or  the  specific  details,  there  are  three  concepts  central  to  all  Student 
Team  Learning  methods:  team  rewards,  individual  accountability,  and  equal  opportunities  for  success. 

Team  rewards  may  consist  of  certificates,  other  forms  of  recognition,  or  grades.  Teams  do  not  compete  for 
limited  rewards.  If  a team  reaches  a certain  criterion,  it  receives  the  reward.  Teams  with  equivalent  results  receive  the  same 
reward.  Individual  accountability  ensures  that  the  team’s  success  depends  on  each  of  the  individual  team  members. 
Knowing  this,  the  team  focuses  its  activity  on  helping  and  tutoring  one  another  to  learn  the  material.  Equal  opportunities 
for  success  is  accomplished  through  the  manner  in  which  rewards  are  given  for  a student’s  having  improved  upon  his  or 
her  past  performance.  Because  high-,  average-,  and  low-achievers  are  equally  challenged  to  do  their  best,  the  contribution 
of  each  team  member  is  of  equal  value. 


Cooperative  Learning  Outcomes 

After  an  analysis  of  over  70  research  studies  of  cooperative  learning,  Slavin  (1989)  concluded  that  cooperative 
learning  is  an  effective  alternative  to  traditional  teaching  methods.  Of  63  studies  in  which  academic  achievement  was 
measured,  57%  found  significantly  greater  achievement  than  control  groups,  while  41%  found  no  statistical  differences, 
and  only  one  study  found  the  control  group  outscoring  the  cooperative  learning  group.  When  Slavin  limited  the  studies  to 
those  in  which  Student  Team  Learning  principles  were  followed,  he  further  concluded  that  34  out  of  41  such  studies 
(83%)  found  significantly  positive  achievement  results. 

An  underlying  aspect  of  cooperative  learning  is  a building-social-skills  component.  By  designing  appropriate 
activities,  the  cooperative  learning  teacher  can  teach  social  skills  to  the  students  at  the  same  time  that  content  is  being 
taught  (Lyman,  Foyle,  & Azwell,  1993).  Haring-Smith  (1993)  pointed  out  that  real-world  jobs  require  workers  to 
cooperate.  She  went  on  to  assert  that  cooperative  learning  teaches  students  how  to  work  together  in  a group,  instead  of 
competing  against  each  other.  Further,  cooperative  learning  students  develop  “considerable  commitment  and  caring  for 
each  other,’’  according  to  Johnson  and  Johnson  (1991). 

Additionally,  Slavin  (1990)  asserted  that  cooperative  learning  improves  intergroup  relations,  encourages 
acceptance  of  mainstreamed  academically-handicapped  students,  builds  self-esteem,  and  improves  time  on  task. 

Computer-Mediated  Cooperative  Learning 

Traditionally,  cooperative  learning  assumes  that  the  group’s  members  are  working  together  in  the  same  location 
and  at  the  same  time  under  the  watchful  eye  of  the  instructor.  The  system  proposed  here — Computer-Mediated 
Cooperative  Learning  (CMCL) — allows  the  students  and  teacher  to  be  remote  from  one  another  in  both  space  and  time. 
As  proposed  here,  CMCL  draws  on  the  dictates  of  cooperative  learning,  while  using  computers  to  provide  a true 
cooperative  learning  experience.  As  described  below,  CMCL  assumes  that  the  computers  will  be  networked  with  e-mail 
capabilities  and  access  to  the  Internet.  There  are  two  components  or  modules  to  CMCL,  one  for  the  teacher  and  another 
for  the  student.  The  teacher’s  module  provides  access  to  tools  that  enable  the  teacher  to  set  up,  monitor,  and  supervise  the 
groups  or  teams.  Similarly  the  students’  module  provides  tools  that  enable  students  to  communicate  with  fellow  grcup 
members  and  the  teacher,  establish  group  goals,  monitor  the  group’s  progress,  access  remote  Internet  sites,  and  prep  are 
the  group’s  final  product. 

Because  CMCL  may  or  may  not  take  place  within  a traditional  classroom  setting,  the  number  of  students  per 
group,  as  well  as  the  number  of  groups,  may  not  seem  to  be  as  constrained  as  in  traditional  cooperative  learning. 
However,  for  the  same  reasons  that  traditional  cooperative  learning  limits  the  number  of  students  per  group  to  six  or 
fewer,  it  is  suggested  that  the  size  of  CMCL  groups  follow  the  same  guidelines.  The  number  of  groups  may  not  be  as 
important  a consideration,  except  that  a large  number  of  groups  working  simultaneously  may  have  an  impact  on  the 
teacher’s  ability  to  monitor  each  group’s  progress  sufficiently. 

The  Teacher’s  Screen 

As  seen  in  figure  1,  the  teacher’s  screen  is  divided  horizontally  into  three  basic  areas:  actions,  options,  and 
information. 
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FIGURE  1 

See  Appedix  C of  the  1996  AECT  Proceedings 

As  the  cooperative  learning  project  proceeds,  there  are  essentially  three  different  activities  the  teacher  must 
complete:  preparing  the  assignment  for  the  students,  monitoring  their  progress,  and  communicating  with  them  on  an 
ongoing  basis.  On  occasion,  the  teacher  may  also  need  context-specific  help  about  the  various  functions  of  the  CMCL 
system.  Hence  there  are  buttons  in  the  action  area  for  each  of  these  activities.  Once  an  action  has  been  selected,  there  are 
then  several  subordinate  actions  to  be  chosen  from;  hence  the  options  area.  The  entire  right-hand  side  of  the  screen  is 
devoted  to  an  area  for  providing  information,  both  textual  and  graphical,  relative  to  each  of  the  actions  and  options. 

Below  the  information  area  is  a control  panel  providing  additional  buttons  for  processing  the  information. 

In  addition  to  the  three  areas  discussed  above,  the  teacher’s  screen  also  has  a message  alert  indicator  in  the  upper 
left  comer  and  a calendar/clock  indicator  in  the  upper  right  comer.  Details  and  possible  functions  of  these  two  indicators 
will  be  described  later. 

Preliminary  preparations.  Before  the  CMCL  experience  can  begin,  the  teacher  must  first  decide  upon  overall 
general  goals  for  the  groups  (see  figure  2).  The  teacher  first  composes  specific  directions  in  terms  of  the  mechanics  of  the 
activity:  the  length  and  format  of  the  final  product,  due  date(s),  grading  procedures,  and  the  like.  Additionally,  the  teacher 
prepares  suggestions  for  the  students  to  use  when  they  are  in  the  various  phases  of  the  activity  (planning,  scheduling, 
preparing  and  revising  drafts,  examining  Internet  sites,  and  publishing  the  final  report).  Because  the  preceding  preparatory 
work  may  be  generic  in  nature,  knowledge  of  the  specific  topic  may  not  be  needed.  Indeed,  many  of  the  suggestions  could 
be  incorporated  into  the  software  package  itself. 

FIGURE  2 

See  Appedix  C of  the  1996  AECT  Proceedings 

The  next  step  in  the  preparation  stage  is  to  choose  a topic  for  the  group  (CMCL  permits  the  teacher  to  assign  a 
different  topic  to  each  group  if  desired).  At  this  point,  the  teacher  may  want  to  add  to  some  of  the  directions  listed  above, 
making  them  more  topic  specific  or  more  student  specific.  For  example,  depending  on  the  topic,  the  teacher  may  have 
specific  suggestions  for  dividing  the  material  into  subtopics  for  individual  team  members.  The  teacher  may  want  to  tell  a 
particular  group  to  be  sure  to  include  this  area  or  avoid  that  area,  or  he  or  she  may  want  to  provide  specific  instructions 
for  a particular  student  based  on  prior  knowledge  of  that  student’s  interests  and  capabilities.  The  teacher  may  also  want  to 
explore  the  Internet  to  find  appropriate  sites  for  a particular  group  to  access.  As  seen  in  figure  2,  the  options  area  of  the 
teacher’s  screen  contains  buttons  to  provide  the  teacher  with  the  tools  necessary  to  compose  directions  and  suggestions 
for  the  students.  CMCL  would  then  make  all  of  this  information  available  to  the  students,  possibly  by  downloading  it  to 
each  student’s  computer,  from  a central  server,  or  across  the  Internet. 

Team  building . After  the  preparations  as  described  above  have  been  made,  the  teacher  is  ready  to  build  the  teams. 
Having  entering  the  students’  names  (and  other  student-specific  information  if  desired)  and  specifying  the  number  of 
teams,  the  teacher  is  presented  with  the  screen  shown  in  figure  3. 

The  options  area  now  contains  a list  (scrolling  if  necessary)  of  the  students,  and  the  information  area  contains  a 
rectangular  box  or  container  for  each  group  (again  scrolling  if  necessary).  The  teacher  drags  a name  from  the  list  into  the 
appropriate  box,  at  which  time  the  name  is  removed  from  the  list  in  the  options  area.  This  operation  proceeds  until  all 
the  names  have  been  placed  in  containers.  It  is  also  possible  to  drag  a name  from  one  container  to  another.  When  the 
teacher  has  built  the  teams,  he  or  she  clicks  on  the  “done”  button  in  the  control  panel  area.  Appropriate  messages  would 
warn  the  teacher  if  any  students  hadn’t  been  placed  in  a group,  or  if  the  pre-specified  group  size  had  been  exceeded  (in 
either  case,  the  teacher  would  have  the  option  of  overriding  CMCL’s  concern). 

FIGURE  3 

See  Appedix  C of  the  1996  AECT  Proceedings 

If  the  appropriate  information  were  available,  CMCL  may  also  provide  options  for  displaying  individual  team 
members  grades  and  the  team’s  grade  average  to  date.  Different  colors  may  be  used  to  indicate  whether  a particular  student 
is  a high-,  middle-,  or  low-achiever.  With  all  this  information  readily  available  and  easily  displayed,  the  teacher  could 
manipulate  the  groups’  composition  to  guarantee  the  heterogeneity  dictated  by  most  traditional  cooperative  learning 
constructs. 
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Message  center.  Because  communication  among  team  members  and  between  the  team  and  the  teacher  is  so 
vitally  important  for  cooperative  learning,  CMCL  has  a message  center  built  into  it.  Both  the  teacher  and  the  students 
have  access  to  the  message  center  for  sending  and  receiving  messages.  The  teacher’s  message  center  (see  figure  4)  uses  the 
options  area  to  display  a list  of  the  teams  and  their  members. 

FIGURE  4 

See  Appedix  C of  the  1996  AECT  Proceedings 

This  list  is  expandable  and  collapsible  so  that  lists  of  one  or  more  teams’  members  is  displayed,  or  only  the  list 
of  teams  is  displayed.  By  clicking  on  an  individual  student’s  name,  the  teacher  is  presented  with  a list  of  messages 
received  from  and  sent  to  that  student.  Previously-read  messages,  as  well  those  to  which  a reply  had  been  sent  may  be 
indicated  (using  symbols  or  colors).  Double-clicking  on  a particular  message  line  displays  the  entire  message.  Similar  to 
the  functions  of  most  e-mail  systems,  the  buttons  in  the  control  panel  at  the  bottom  enable  the  teacher  to  compose  new 
messages,  reply  to  received  messages,  forward  messages  to  other  students,  print  messages,  archive  or  delete  messages,  and 
so  forth.  The  teacher  also  has  the  option  of  selecting  more  than  one  student  or  even  entire  teams  from  the  list  for  the 
purpose  of  sending  the  same  message  to  more  than  one  student  at  a time.  It  is  also  through  the  message  center  that  the 
teacher  receives  drafts  and  final  reports  from  the  teams. 

Activity  reports . Some  cooperative  learning  methods  such  as  Group  Investigation  require  that  the  teacher 
evaluate  an  individual  student’s  achievement  by  examining  the  student’s  higher-order  thinking  skills  (Sharan  & Sharan, 
1976).  By  overseeing  the  students  as  they  work  in  their  groups,  the  teacher  may  observe  and  evaluate  the  student’s  work 
habits,  grasp  of  the  subject,  thinking  processes,  and  the  like.  With  students  working  individually  at  computers  possibly 
remote  from  the  teacher,  such  evaluation  is  decidedly  more  complicated.  It  is  for  this  reason  that  CMCL  has  an  “activity 
reports”  function  built  in  (see  figure  5). 


FIGURE  5 

See  Appedix  C of  the  1996  AECT  Proceedings 

As  students  go  about  their  various  activities,  CMCL  keeps  track  of  what  the  students  are  doing  on  the 
computer.  A built-in  monitoring  system  records  which  activity  the  student  has  selected  (from  the  actions  and  options 
available  to  him  or  her),  the  date  of  the  activity,  and  the  length  of  time  spent  on  that  activity.  When  appropriate,  CMCL 
records  additional  information  such  as  the  uniform  resource  locator  (URL)  address  for  any  Internet  site  accessed,  details  for 
any  messages  sent  and  received,  contents  of  the  student’s  and  team’s  notebooks  (discussed  in  detail  later),  and  so  forth. 

When  the  teacher  selects  “Activity  Reports,”  he  or  she  is  given  the  same  expandable/collapsible  list  as  before 
with  a listing  for  each  team  and  each  student.  Clicking  on  a student’s  name  displays  a summary  report  of  that  student’s 
activity  which  can  then  be  expanded  further  to  reveal  the  individual  activities.  The  teacher  can  then  select  an  activity  from 
that  list  and  examine  any  additional  details  in  the  report  (by  clicking  on  the  “Details”  button  in  the  control  strip).  Armed 
with  the  content  and  sequence  of  a student’s  activity,  the  teacher  should  be  better  able  to  evaluate  that  student’s  progress 
at  any  stage  of  the  cooperative  learning  assignment.  Granted,  the  “time  spent”  as  recorded  in  an  activity  report  is  not 
necessarily  time  on  task,  but  the  teacher  does  at  least  know  that  the  student  did  indeed  engage  in  that  activity  for  some 
length  of  time.  It  is  hoped  that  the  entire  scope  of  a student’s  activities  will  form  an  accurate  picture  of  what  the  student 
has  done. 

The  Student’s  Screen 

The  layout  and  general  function  of  the  student’s  screen  is  similar  to  that  for  the  teacher  (see  figure  6).  It  too  is 
divided  horizontally  into  the  same  three  areas:  actions,  options,  and  information. 

FIGURE  6 

See  Appedix  C of  the  1996  AECT  Proceedings 

Message  center.  The  student’s  message  center  functions  in  a manner  similar  to  that  for  the  teacher.  Messages  can 
be  sent  to  and  received  from  fellow  team  members  and  the  teacher.  Depending  on  the  nature  of  the  assignment,  the  teacher 
can  also  enable  messaging  between  members  of  different  teams  as  well  (since  some  forms  of  cooperative  learning  support 
intergroup  collaboration,  while  others  do  not). 
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Examine  assignment.  With  this  function,  the  student  is  able  to  examine  the  preparatory  information  that  the 
teacher  has  supplied  (see  figure  7). 


FIGURE  7 

See  Appedix  C of  the  1996  AECT  Proceedings 

A collapsible/expandable  list  of  subjects  appears  in  the  option  area.  Clicking  on  any  one  displays  the  teacher- 
prepared  material  in  the  information  area.  It  is  here  that  the  student  has  access  to  the  generic  instructions,  as  well  as 
specific  instructions  directed  toward  the  topic,  the  team,  or  the  individual  student.  The  control  panel  contains  buttons  for 
processing  the  information  (printing,  saving,  and  the  like). 

Plan  & schedule.  Perhaps  no  other  area  of  traditional  cooperative  learning  relies  as  much  on  the  team  members’ 
being  together  as  does  the  planning  and  scheduling  phase  of  the  project.  Relying  on  an  awareness  of  each  other’s 
strengths  and  weaknesses,  team  members  working  in  person  can  negotiate  precisely  who  is  going  to  do  what  and  when. 
Because  the  team  members  in  CMCL  may  or  may  not  know  each  other,  may  or  may  not  be  able  to  physically  meet  as  a 
group,  CMCL  relies  on  the  messaging  system  for  planning  and  scheduling.  After  one  student  is  selected  from  the  group, 
either  by  the  teacher  beforehand  or  by  the  group  via  the  message  center,  the  teacher  activates  the  planning  and  scheduling 
controls  for  that  student  alone  (see  figure  8). 


FIGURE  8 

See  Appedix  C of  the  1996  AECT  Proceedings 

After  selecting  "plan”  from  the  options  area,  the  designated  student  is  presented  with  a list  of  tasks  to  be 
completed  by  the  group.  The  teacher  will  have  already  prepared  a list  of  required  tasks,  as  well  as  list  of  optional  tasks. 
After  consultation  with  the  other  team  members  (via  the  message  center),  the  designee  may  add  additional  tasks  to  the  list 
by  means  of  keyboard  entry.  These  three  different  kinds  of  tasks  (required,  optional,  and  student-generated)  may  be 
signified  by  different  colors  for  ease  of  identification.  The  student  uses  a click-and-drag  technique  (similar  to  that  that  the 
teacher  had  previously  used  to  assemble  the  groups)  to  place  the  tasks  into  the  team  members’  boxes.  As  before,  tasks 
can  be  moved  from  one  box  to  another.  It  is  assumed  that  this  division  of  labor  is  accomplished  after  considerable 
discussion  (via  the  message  center)  among  the  group’s  members.  (An  alternative  to  having  a designated  student  assign  the 
tasks  would  be  to  have  the  teacher  fill  this  role.  This  functionality  would  then  have  to  be  added  to  the  teacher’s  module.) 

After  the  tasks  have  been  assigned,  the  student  clicks  on  the  "Done”  button  in  the  control  strip  and  the  task 
assignments  are  recorded  by  the  system  and  made  accessible  to  the  other  team  members.  (If  any  required  tasks  had  not  yet 
been  assigned,  CMCL  would  respond  with  an  appropriate  message.)  The  other  team  members  then  have  the  ability  to 
look  at  the  task  assignments,  but  not  alter  them  (prior  to  that  time,  any  attempt  by  the  other  team  members  to  access  the 
planning  windows  would  be  met  with  an  "in  progress”  message).  The  original  designee  retains  the  ability  to  alter  task 
assignments  (move,  add,  and  remove),  with  the  agreement  of  the  group  of  course.  The  list  of  task  assignments  is 
available  to  the  teacher  through  the  activity  report  for  the  team. 

Scheduling  proceeds  in  a similar  manner.  When  a student  selects  the  scheduling  option,  his  or  her  list  of  tasks  is 
displayed  in  the  options  area.  The  information  area  displays  a calendar  with  the  teacher-imposed  deadlines.  The  student 
then  drags  each  task  to  a box  in  the  calendar.  Attempts  to  place  tasks  beyond  relevant  deadlines  is  met  with  appropriate 
warning  messages.  As  with  planning,  the  student  can  choose  to  alter  the  schedule  at  a future  time.  A student’s  schedule  is 
available  to  the  teacher  through  that  student’s  activity  report. 

Examine  sites.  The  heart  of  CMCL  is  the  student’s  ability  to  examine  sites  on  the  Internet.  By  clicking  on  the 
"Examine  Sites”  button,  the  student  is  given  two  options:  searching  the  Internet  or  looking  at  teacher-suggested  sites  (see 
figure  9). 


FIGURE  9 

See  Appedix  C of  the  1996  AECT  Proceedings 

CMCL  can  provide  either  a keyword  search  or  a content  search,  possibly  using  the  search  engines  and  content 
lists  already  available  on  the  Internet.  The  interface  for  this  activity  is  similar  to  that  used  by  such  popular  browsers  as 
Netscape  and  Mosaic.  (Although  students  cannot  easily  be  prevented  from  conducting  irrelevant  searches,  the  absence  of 
buttons  such  as  "What’s  Hot”  and  "Cool  Sites”  may  help.)  Clicking  on  "Suggested  Sites”  displays  the  sites  that  the 


679 


3'J3 


teacher  had  previously  entered  into  the  system.  Again,  the  display  is  similar  to  the  list  generated  by  a search  in  Netscape 
or  Mosaic.  Clicking  on  any  listing  opens  the  URL  for  that  listing. 

The  “Save  Site”  button  in  the  control  strip  allows  the  student  to  save  the  site’s  URL  to  the  student’s  personal 
notebook  for  future  reference.  While  examining  a site,  a student  can  select  text  or  graphics  to  be  placed  in  the  student’s 
personal  notebook  as  well.  Clicking  on  the  “Save  Item”  button  copies  the  selected  text  or  graphic  to  the  notebook, 
downloading  the  file  if  necessary. 

Prepare  draft.  After  the  student  has  gathered  information  from  the  Internet,  it  is  time  to  prepare  a draft  of  his  or 
her  portion  of  the  final  project  (see  figure  10).  Both  the  information  gathered  and  the  draft  itself  are  saved  in  the  student’s 
personal  “notebook.” 

The  “Site  List”  button  permits  the  student  to  easily  return  to  the  previously-saved  URL’s  on  the  Internet  at  the 
click  of  the  mouse.  The  “Saved  Items”  button  functions  as  a “library”  of  saved  items.  As  with  libraries  in  many  software 
programs,  a CMCL  library  can  be  viewed  an  individual  item  at  a time  or  as  a scrolling  list  of  so-called  “thumbnails” 
(abbreviated  versions  of  text  items  and  miniature  representations  of  graphic  items).  Clicking  on  a thumbnail  displays  the 
complete  item. 

The  “Draft”  button  enables  the  student  to  assemble  the  results  of  his  or  her  efforts  into  a document.  Depending 
on  the  sophistication  and  abilities  of  the  students,  as  well  as  the  nature  of  the  final  product  (as  determined  by  the  teacher), 

the  draft  mode  may  function  like  an  integrated  software  package  such  as  Microsoft  Works®,  as  a presentation  program 
such  as  PowerPoint®,  as  a page  layout  program  such  as  PageMaker®,  or  as  a web  page  creation  program  such  as 
PageMill®,  for  example.  Ideally,  this  functionality  would  be  built  into  the  CMCL  system,  but  realistically,  it  may  be 
more  efficient  to  link  to  a dedicated  program  instead.  Regardless  of  which  route  is  taken,  the  student  should  have  full  use 
of  appropriate  tools  for  composing  and  editing  text  and  graphics. 

FIGURE  10 

See  Appedix  C of  the  1996  AECT  Proceedings 

Upon  completion,  the  student  can  send  the  draft  to  fellow  team  members  and  the  teacher  for  comment.  Using  an 

annotation  system  similar  to  that  in  Microsoft®  products,  the  draft  .^viewer  can  attach  collapsible  Post-it™-like  notes  to 
the  draft , which  can  then  be  expanded  upon  receipt  by  the  student  who  prepared  the  draft.  As  computer  systems  attain 
increased  speed  and  memory,  such  annotation  might  indue*  digitized  audio  and  video  from  the  reviewer. 

Publish  report.  As  with  the  planning  phase  of  the  project,  the  publishing  of  the  final  report  is  prepared  by  one 
member  of  the  group,  determined  in  advance  during  task  assignment.  As  might  be  expected,  this  student  takes  the  revised 
drafts  from  the  team  members  ai  1 assembles  them  into  a coherent  whole  using  the  same  document  preparation  package  as 
before.  Once  again,  the  draft  of  the  final  product  is  shared  with  the  team  members  and  the  teacher  for  comment,  using  the 
same  annotation  system  as  before.  The  culmination  of  this  cooperative  learning  experience  is  sharing  the  final  product 
with  the  rest  of  the  teams  via  the  network. 

Miscellaneous  Features 

The  “message  alert”  indicator  in  the  upper  left  comer  of  both  the  teacher’s  and  student’s  screens  may  be 
configured  to  function  in  several  ways.  For  example,  it  may  display  the  number  of  unread  messages,  it  may  flash  to 
indicate  that  a message  has  just  been  received,  or  it  may  turn  a certain  color  to  indicate  that  there  is  an  unread  message 
from  the  teacher. 

The  “date  and  time  indicator”  may  likewise  be  used  to  display  pertinent  information  for  a student.  For  example, 
clicking  on  it  may  display  the  amount  of  time  since  the  person  first  logged  on  for  the  current  session,  it  may  display  the 
total  amount  of  time  that  the  person  has  spent  on  the  project,  or  it  may  turn  a progressively  more  intense  color  as  a 
deadline  approaches. 

The  “Help”  button  should  be  context  sensitive.  It  may  be  configured  to  provide  so-called  “balloon  help”:  as  the 
user  drags  the  cursor  around  the  screen,  cartoon-like  balloons  appear  explaining  the  function  and  features  of  the  item  the 
cursor  is  currently  over.  Help  may  also  be  set  up  to  display  a list  of  topics  that  are  applicable  to  the  user’s  current 
situation.  Clicking  on  any  item  in  the  list  provides  further  details. 

Likely  Problems 

The  hardware  and  software  requirements  for  CMCL  do  not  come  cheaply.  It  is  expected  that  each  student  will 
have  the  availability  of  a fast  machine  with  a large  monitor  (17<  minimum)  and  connections  to  a local  area  network,  a 
high-quality  printer  (preferably  color),  and  the  Internet.  As  with  many  computer- mediated  projects,  the  technology  often 
takes  center  stage  when  problems  arise:  things  aren’t  connected  properly,  equipment  simply  doesn’t  work,  network 
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connections  are  lost  or  disabled  entirely,  students  don’t  know  how  to  operate  certain  functions  of  the  computer  or  the 
software,  and  so  forth.  Consequently,  the  availability  of  a technician  familiar  with  both  the  hardware  and  software  is 
desired. 

In  light  of  recent  publicity  about  the  availability  of  pornography  on  the  Internet  (Elmer-DeWitt,  1995;  Rimm, 
1995),  many  teachers,  parents,  and  students  are  concerned — -justifiably  or  not — about  this  aspect  of  using  the  Internet.  It 
is  therefore  assumed  that  the  teacher  will  at  the  very  least  disable  searches  using  words  in  a predefined  list  of 
objectionable  terms.  It  is  further  hoped  that  the  teacher  will  use  one  of  the  emerging  commercial  products  for  blocking 
access  to  objectionable  sites. 

The  right  of  privacy  may  be  another  potential  concern  for  some  students.  Because  the  teacher  will  be  able  to 
look  at  a log  of  each  student’s  activities  (sites  visited,  e-mail  messages  sent,  and  the  like),  it  is  important  that  students 
(and  their  parents)  be  made  well  aware  of  this  feature  at  the  outset.  However,  the  student  must  not  be  made  to  feel  that 
“Big  Brother”  is  watching.  The  teacher  has  a legitimate  concern  for  wanting  to  see  this  information;  he  or  she  needs  such 
information  to  accurately  judge  a student’s  intellectual  growth  and  academic  achievement. 

Because  students  may  not  have  the  direct,  in-person  supervision  of  the  teacher,  CMCL  may  be  set  up  so  that 
students  are  required  to  complete  their  work  in  a logical  sequence.  For  example,  they  can’t  prepare  the  final  project  before 
they  have  divided  the  tasks  and  visited  some  sites.  CMCL  can  be  configured  so  that  certain  functions  are  not  available 
(indicated  by  dimmed  buttons,  for  example)  until  certain  other  functions  have  been  given  enough  attention  (assignment  of 
all  the  tasks,  a pre-specified  amount  of  time  spent  on  a certain  activity,  or  a pre-specified  number  of  sites  visited,  for 
example). 

Performing  a certain  activity  within  CMCL  may  be  more  time  consuming  than  conducting  that  same  activity  in 
an  actual  face-to-face  meeting.  For  example,  dividing  the  task  assignments  requires  a fair  amount  of  negotiation,  and 
sending  e-mail  messages  back  and  forth  can  slow  down  this  process  considerably.  If  the  students  are  all  in  the  same 
classroom,  there  is  no  reason  why  they  can’t  complete  some  activities  in  person  before  entering  the  information  into  the 
system. 


Closing  Comments 

CMCL  requires  both  teachers  and  students  to  rethink  how  teaching  and  learning  take  place.  It  puts  an  additional 
burden  on  the  teacher,  particularly  in  terms  of  preparation  and  evaluation.  Drawing  upon  the  research  on  cooperative 
learning  and  Student  Team  Learning  in  particular,  CMCL  has  the  same  potential  for  enhancing  the  learning  experience. 

In  addition  to  using  certificates  and  the  like  for  team  rewards,  the  teacher  may  also  publish  the  best  final  products  on  the 
Internet.  Because  each  student  is  responsible  for  producing  his  or  her  own  part  of  the  final  document,  individual 
accountability  is  maintained.  Finally,  because  the  teacher  can  monitor  an  individual  student’s  improvement  and  offer 
guidance  when  necessary,  CMCL  provides  all  students  with  equal  opportunities  for  success. 

Since  cooperative  learning  relies  so  much  on  the  interpersonal  relationships  among  a group’s  members,  it  is 
reasonable  to  question  whether  or  not  learners  seated  at  their  own  computers — possibly  remote  from  one  another,  as  well 
as  remote  from  the  teacher — can  have  fie  same  rich,  social  experience  that  a group  seated  at  the  same  table  can  have. 
CMCL,  especially  through  the  use  of  the  message  center  and  the  activity  reports  which  enable  the  students  and  teacher  to 
be  in  constant  contact  with  one  other,  develops  a new  kind  of  group  dynamic,  but  one  which  has  the  potential  for  an 
engaging  and  productive  social  experience. 
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The  body  of  research  literature  on  learning  from  film  and  television  is  voluminous.  There  are  many  documents 
which  synthesize  specific  aspects  of  this  literature,  but  none  which  cover  both  major  streams  of  research,  mass  media  and 
instructional  television.  This  study  reports  a major  synthesis  effort  that  integrates  both  sources  of  research.  The  body  of 
literature  reviewed  encompasses  17,500  citations  from  the  Educational  Resources  Information  Clearinghouse  (ERIC)  and 
1 ,882  citations  from  Psychological  Abstracts. 

The  methodology  selected  for  this  review  is  integrative  research  as  described  by  Harris  M.  Cooper  (1989).  This 
methodology  was  selected  because  it  enables  the  researcher  to  summarize  past  research  by  drawing  conclusions  from 
many  separate  and  diverse  studies.  This  method  requires  completing  five  stages  in  such  a manner  as  to  ensure  that 
sources  of  invalidity  are  controlled. 

The  first  stage  is  problem  formulation  in  which  definitions  are  formed  which  distinquish  relevant  and  irrelevant 
studies.  The  major  source  of  error  at  this  stage  is  not  starting  broadly  enough  and  focusing  too  early  on  narrow  concepts 
and  superficial  details.  The  second  stage  is  data  collection  in  which  sources  of  potentially  relevant  studies  are  identified. 
The  main  sources  of  invalidity  at  this  stage  are  that  studies  and  samples  may  differ  from  the  target  population,  in  the  case 
the  body  of  literature  identified.  The  third  stage  is  data  evaluation  in  which  criteria  are  formulated  and  applied  to 
distinquish  valid  from  invalid  studies.  At  this  point  invalidity  can  arise  from  inclusion  of  studies  with  flawed  research. 
The  fourth  stage,  analysis  and  interpretation,  deals  with  synthesizing  the  body  of  literature  identified  as  representative  and 
methodologically  defensible.  Invalidity  can  result  from  unwarranted  inferences  from  extraneous  data  or  variables  or  from 
synthesis  assumptions  that  are  wrong.  The  fifth  stage  is  presentation  which  allows  the  researcher  to  use  visuals  to 
describe  logical  relationships  across  the  data.  The  sources  of  error  may  include  omission  of  review  procedures  or 
findings.  The  use  of  visuals  and  presentation  strategies  enhances  the  synthsizing  process. 

A comprehensive  historical  review  of  television  research  revealed  two  parallel  strands  which  evolved  separately. 
The  first  of  these  which  was  related  to  education  and  psychology  began  in  the  1950's  with  comparative  media  research, 
evolved  into  media  effects  and  individual  differences  and  is  represented  today  by  research  on  interactions  from  viewer 
characteristics  and  television  effects.  The  parallel  strand  originated  during  the  1960's  in  research  related  to  mass  media  and 
social  effects  and  formal  features. 

This  historical  review  yielded  the  major  categories  of  the  television  viewing  system.  This  television  viewing 
system  concept  was  present  in  the  literature  as  a way  to  explain  the  totality  of  the  television  viewing  experience. 
Research  evolved  around  the  three  components  of  this  viewing  system:  programming,  environment,  and  behavior. 

Figure  1 , TV  Viewing  System,  illustrates  the  components  of  the  viewing  and  the  research  areas  associated  with  each. 
The  programing  and  environment  components  represent  independent  and  mediating  variables  respectively;  while  behavior 
represents  dependent  variables. 


See  Appendix  D of  the  1996  AECT  Proceedings 
Figure  1.  TV  Viewing  System 

After  the  literature  was  analyzed  from  the  point  of  view  of  the  television  viewing  system,  a number  of  major 
research  areas  became  apparent.  These  were: 

•message  design  and  cognitive  processing 
•school  achievement 
•family  viewing  context 
•attitudes,  beliefs,  behaviors 
•programming  and  utilization 
•critical  viewing  skills. 

The  problem  was  reformulated  as  reviewing  and  synthesizing  research  in  these  areas. 

The  next  step  was  data  collection.  Classic,  representative,  and  pivotal  studies  in  each  area  were  identified 
through  review  books  and  documents,  computerized  searching,  and  indexing  of  relevant  articles  from  journals  in  the  field 
from  the  1960’s  on.  Original  documents  were  obtained  and  studied.  Extensive  files  in  each  area  were  organized,  and  cross 
referencing  was  done. 

Each  of  the  authors  of  this  paper  choose  different  areas  in  which  to  begin  the  next  step,  data  evaluation.  In 
addition  to  summarizing  findings,  each  author  was  to  identify  methodological  problems  in  individual  studies  and  across 
the  literature.  Seriously  flawed  or  inconsequential  studies  were  withdrawn  from  the  population  of  studies. 

Several  methodological  problems  were  identified  across  the  research  areas.  Operational  definitions  varied 
between  psychologists  and  educators  and  even  between  individual  psychologists  or  between  individual  educators. 


Conclusions  of  laboratory  studies  were  inferred  to  apply  to  realistic  settings.  Confounding  variables  in  field  and  home 
settings  were  not  controlled  or  in  some  cases  even  recognized.  Different  variables  were  studied  at  different  ages  which 
made  comparison  and  synthesis  impossible  beyond  one  age  level.  There  was  an  over-reliance  on  self-reporting  methods 
throughout  most  of  the  research. 

When  data  evaluation  in  each  of  the  areas  was  complete,  an  analysis  and  interpretation  was  done  for  each  area  and 
then  for  the  whole  study.  Conclusions  for  each  area  were  summarized,  and  recommendations  for  future  research  were 
prepared.  Then,  the  authors  analyzed  and  interpreted  the  data  of  the  whole  study.  To  do  this  several  assumptions  were 
made: 


•causality  statements  must  be  well-supported 
•substantive  conclusions  from  each  area  should  remain  separate 
•methological  conclusions  from  each  area  could  be  integrated,  and 
•graphic  comparisons  among  areas  would  be  helpful. 

The  authors  began  with  group  editing  of  the  summaries  and  conclusions  then  proceeded  to  graphic  comparison. 

The  first  comparison  was  done  by  listing  the  research-based  conclusions  in  one  column  and  general 
misconceptions  in  the  other  column.  The  second  comparison  required  a table  which  related  the  research  areas  with  the 
specific  variables  addressed  throughout  the  research.  Table  1 , Research  Areas  and  Variables  Addressed  in  Specific  Areas, 
is  the  result  of  this  comparison. 

The  next  graphic  comparison  concerned  the  identification  of  dominant  research  methodologies  in  the  various 
research  areas.  From  this  representation  it  is  clear  that  certain  methodologies  have  been  heavily  used  to  address  a variety 
of  research  areas  while  other  methodologies  appear  not  to  have  been  used  in  an  area.  The  areas  where  these  other 
methodologies  have  not  been  used  may  provide  fruitful  research  opportunities  which  would  strengthen  the  research  in 
each  area.  Few  research  areas  have  benefited  from  the  use  of  all  methodological  approaches.  This  comparison  is  shown 
in  Table  2,  Research  Methodologies  Employed  in  the  Various  Research  Areas. 


Table  1 Research  Areas  and  Related  Variables 
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Research 

Areas 

Early  Studies  1 

Message  I 

Design  1 

Cognitive  1 

Processing  1 

School  1 

Achievement  1 

Imagination  1 

Viewing 

Environment 

Co  viewing  1 

Viewing  1 

Habits  1 

Attitudes  I 

Beliefs  J 

Behavior  1 

Programming  | 

Utilization  1 

Critical 

Viewing 

Table  2 Methodological  Approaches  and  Research  Areas 

The  last  graphic  technique  used  for  synthesis  was  employed  to  identify  the  relationship  of  instructional 
technology  practioners  to  application  of  the  research  on  learning  from  television.  The  technique  used  was  to  separate  in 
boxes  the  roles  assumed  and  the  tasks  subsumed  under  role.  This  analysis  is  presented  in  Figure  2,  Roles  of  the 
Instructional  Technologist. 


See  Appendix  D of  the  1996  AECT  Proceedings 

Figure  2.  Roles  of  the  Instructional  Technologist  ' 

The  last  stage  of  the  integrative  research  process  was  public  presentation.  The  presentation  of  the  findings 
highlighted  selected  conclusions  in  each  of  the  major  research  areas:  message  design  and  cognitive  processing,  school 
achievement,  family  viewing  context,  attitudes,  beliefs,  and  behaviors,  programming  and  utilization,  and  critical  viewing 
skills.  The  presentation  included  an  explanation  of  the  research  process  used  in  this  study.  In  addition,  methodological 
sources  of  error  were  identified  and  summarized.  Future  directions  for  research  related  to  television  effects  were  presented. 
These  were: 

•examine  long  term/cumulative  effects 

•conduct  more  naturalistic  studies 

•research  effects  of  prosocial  programming 

•identify  societal  variables  which  interact  with  television  effects 

•encourage  causal  research  related  to  television  and  behavior. 

The  complete  results  of  the  study  will  be  published  as  Chapter  1 1 , Research  on  Learning  from  Television  in  Handbook 
of  Research  on  Educational  Communications  and  Technology  (in  press). 
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Every  day  we  are  bombarded  with  messages.  Our  minds  sift  through  the  thousands  of  signals  we  receive  and  chooses 
which  ones  we  pay  attention  to,  which  ones  we  remember  for  a short  period  of  time,  and  which  ones  go  into  our  long- 
term memory.  Such  message  bombardment  exists  in  the  form  of  advertisements  on  a radio,  posters  on  a bus,  news  on 
television,  directions  on  a box  of  brownie  mix,  letters  retrieved  from  e-mail,  and  a warning  on  an  antihistamine  package. 
The  mind  must  sift  well  in  order  to  weed  out  the  many  other  messages  we  receive  that  are  unimportant. 

It  is  not  surprising,  therefore,  that  when  a message  is  intended  to  be  instructional,  great  care  must  go  into  planning 
for  the  content  and  form  that  will  attract  attention  and  facilitate  perception,  comprehension,  and  retention.  An 
instructional  designer  or  developer  uses  research  and  theory  to  produce  a message  that  retains  attention  and  results  in 
learning.  To  do  this,  the  designer  or  developer  draws  on  theory  from  many  fields  including: 

psychological  and  learning  theories  which  provide  principles  related  to  outcomes 
communication  and  systems  theories  which  provide  process  models 
aesthetic  principles  which  provide  guidelines  for  graphic  design  and  cuing. 

Since  the  1950's,  one  journal  has  continuously  published  articles  on  message  design.  This  journal  is  Educational 
Technology  Research  and  Theory  formerly  Educational  Communications  and  Technology  Journal  and  Audiovisual 
Communications  Review.  Many  books  have  addressed  the  topic  including  Berelson  & Steiner,  1964;  Fleming  & Levie, 
1978;  1993;  West,  Farmer,  & Wolff,  1991;  and  Tufte,  1983;  1990.  The  Handbook  of  Research  on  Educational 
Communications  and  Technology  (Jonassen,  in  press)  devotes  several  of  its  42  chapters  to  principles  related  to  the 
design  of  messages  including  chapters  on: 

Behaviorism  and  Instructional  Technology 
Cognitive  Perspective  in  Psychology 
Research  on  and  Research  with  Emerging  Technologies 
Visual  Literacy 

Visual  Message  Design  and  Learning 
Text  Design 

Multiple-Channel  Communications 
Instructional  Technology  and  Attitude  Change 

One  consequence  of  these  multiple  sources  of  research  and  theory  is  designers  who  are  overwhelmed  by  an  abundance 
of  principles  and  guidelines  because  they  do  not  have  tools  to  facilitate  taking  many  variables  into  account  concurrently. 
Despite  the  voluminous  research  published  and  the  numerous  models,  concepts,  and  theories  hypothesized,  there  is  little 
to  help  the  designer  understand  and  apply  this  work  because  conceptual  frameworks  which  illuminate  relationships  among 
message  design  variables  are  inadequate. 

Furthermore,  the  art  of  message  design  has  been  practiced  with  the  tools  of  another  age.  Paper  and  pencil  are  fine 
tools,  but  their  use  is  limited.  Working  in  the  field  of  instructional  technology,  one  cannot  avoid  wondering  when  the 
technology  will  be  applied  to  the  field  itself.  Efforts  have  been  made  to  automate  design  including  message  design. 
Richey  (in  press)  reviews  trends  in  research  on  automated  design  tools  and  the  practicality  of  such  tools.  Powell  and 
Okey  (1994)  discuss  the  role  of  performance  support  in  multimedia  authoring.  Merrill  and  his  colleagues  have  developed 
Instructional  Transaction  Theory  which  addresses  instructional  strategies  as  well  as  message  design  (Merrill  & ED2 
Research  Team,  April, 1993;  Merrill,  Jones,  & Li,  June, 1992).  Issues  of  evaluating  automated  design  have  been  raised 
by  Gros  and  Spector  (1994).  Nevertheless,  no  commonly  accepted  automated  procedure  for  message  design  has  been 
produced. 

Our  goal  was  to  develop  a conceptual  structure  that  would  lead  to  an  improved  process  for  message  design  by  making 
relationships  among  variables  clearer;  thus,  theories  from  diverse  sources  would  become  more  manageable.  To  develop 
this  conceptual  structure  and  process,  many  issues  had  to  be  resolved.  The  process  was  evaluated  formatively  as  the 
structure  was  being  developed. 

The  ideas  presented  here  are  seen  as  an  initial  step  in  developing  an  electronic  performance  support  system  (Witt  & 
Wager,  1994;  Gery,  1993;  Stevens  & Stevens,  1995)  for  designing  messages.  When  combined  with  the  work  of  others, 
such  as  the  work  of  Keller  and  others  on  motivation  (Keller  & Burkman,  1993;  McAleese  & Gunn,  1994)  and  the  work 
of  Boling  (1995;  1994)  on  screen  design  principles,  this  conceptual  structure  could  become  part  of  a computerized  tool  to 
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support  message  design.  Even  if  that  progression  does  not  occur,  results  from  formative  evaluations  substantiate  the 
value  of  the  conceptual  structure  as  a base  for  procedural  theory  which  disciplines  and  guides  designers.  The  purposes  of 
this  research  are  (1)  to  develop  a conceptual  framework  for  message  design,  (2)  to  develop  a systematic  procedure  for 
message  design,  and  (3)  to  provide  one  basis  for  the  development  of  a performance  support  system. 

The  Theoretical  Base  for  Message  Design 

Since  the  1960's  researchers  in  instructional  technology  have  pursued  the  goal  of  a theoretical  base  for  message 
design.  This  goal  has  been  elusive  despite  (a)  the  synthesis  of  research  into  principles,  (b)  the  extension  of  principles  in 
areas  such  as  motivation  and  concept  learning,  and  © the  increasing  availability  of  tools  for  the  design  of  messages  and 
researching  message  design.  In  1962  an  issue  of  Audiovisual  Communication  Review  was  devoted  to  the  relationship  of 
perception  theory  to  the  design  of  audiovisual  messages  (Norberg,  1962).  As  early  as  1963  a definition  of  the  field 
described  instructional  technology  as  the  study  and  practice  of  controlling  messages  (Seels  & Richey,  1994).  All  of  the 
articles  in  the  July-August  1963  issue  of  Audiovisual  Communications  Review  addressed  message  design  because  the 
issue  included  articles  on  the  use  of  pictures,  repetition  in  educational  television,  opinion  change  as  mediated  by  film, 
effectiveness  of  films  with  indirect  presentations,  and  pacing  in  programed  instruction. 

Berelson  and  Steiner  (1964)  culled  principles  from  psychology  that  could  help  the  behavioral  scientists  in  a variety 
of  roles.  Fleming  and  Levie  (1978)  extended  the  work  of  Berelson  and  Steiner  but  directed  their  book  towards 
instructional  planners,  such  as  message  designers.  Fleming  and  Levie’s  first  edition  had  a great  impact  on  the  field  as 
documented  by  the  number  of  reviews  (Markle,  1979;  Borich,  1979;  Clark;  1979)  that  appeared  at  that  time  and  the 
awards  presented  to  the  authors  by  the  Association  for  Educational  Communications  and  Technology  and  the  National 
Society  for  Performance  and  Instruction.  The  term  message  design  was  used  in  the  literature  frequently  thereafter. 

Research  in  diverse  areas,  such  as  cue  summation,  concept  formation,  animation,  and  attitude  change,  followed  and 
contributed  to  a growing  knowledge  base.  In  1991  West,  Farmer  and  Wolff  synthesized  principles  from  the  cognitive 
sciences  especially  as  they  related  to  frames  or  visual  representation  of  content  relationships.  The  second  edition  of 
Fleming  and  Levie’s  book  appeared  in  1993.  It  reflected  advances  in  research  and  theory  and  included  chapters  which 
covered  additional  learning  outcomes.  The  second  edition  also  substituted  a chapter  on  learning  strategies  for  principles 
of  memory  learning.  The  second  edition  was  well  received  (Allen,  1993);  although  the  impact  seems  to  have  been  less 
than  with  the  first  edition  based  on  the  number  of  reviews  published. 

Grabowski  (1995)  and  Berry  (1995)  have  written  articles  which  trace  the  evolution  of  message  design  as  an  area  of 
study  and  practice.  Grabowski  reviews  major  contributors  to  theory  about  message  design  and  explains  how  the  concept 
of  mental  processing  or  "inductive  composition"  has  been  added  to  the  concept  of  planning  for  the  physical  form  of  the 
message.  Berry  describes  the  area  of  message  design  as  becoming  increasingly  focused  on  circumscribed,  specific 
messages,  such  as  individual  visuals,  computer-based  instruction  sequences,  and  pages  or  screens.  Thus,  current  research 
is  directed  more  towards  visuals  and  their  processing  than  towards  programs  or  speeches. 

Although  the  concept  of  message  design  has  been  evolving  since  the  1960's,  the  first  problem  encountered  while 
developing  this  conceptual  structure  was  defining  message  design.  Several  questions  about  the  nature  and  scope  of 
message  design  arose: 

What  constitutes  a message? 

What  does  the  message  design  process  encompass? 

What  is  the  difference  between  instructional  strategies  and  message  design? 

We  will  address  each  question  and  its  resolution  in  order. 

Both  editions  of  the  books  by  Fleming  and  Levie  (1978, 1993)  describe  a message  as  "a  pattern  of  signs  (words, 
pictures,  gestures)  produced  for  the  purpose  of  modifying  the  psychomotor,  cognitive,  or  affective  behavior  of  one  or 
more  persons  (1978,  p.  x)."  According  to  Grabowski  (1995),  external  factors  relate  to  message  design  for  instruction  and 
include  dealing  with  attention,  perception,  and  comprehension.  Internal  factors  relate  to  message  design  for  learning  and 
include  dealing  with  retention  and  retrieval. 

What  then  is  message  design?  The  various  definitions  have  in  common  planning  for  the  form  in  which  the  message 
/ should  appear.  According  to  Seels  and  Richey  (1994),  "Message  design  encompasses  principles  of  attention,  perception, 
and  retention  that  direct  specifications  for  the  physical  form  of  the  messages  which  are  intended  to  communicate  between 
a sender  and  a receiver  (p.  31 )."  When  message  design  is  compared  with  instructional  strategy  design,  it  becomes  clear 
that  the  construct  "instructional  strategy"  differs  from  the  construct  "message  design"  by  encompassing  all  the  steps  in 
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lesson  design  rather  than  just  the  ones  related  to  presentations.  "Instructional  strategies  are  specifications  for  selecting  and 
sequencing  events  and  activities  in  a lesson"(Seels  & Richey,  1994,  p.  129).*' 

Figure  1.  Relationship  of  Messages  to  Instructional  Strategy, 

Instructional  Strategy 

Presentation  Practice  Feedback 

Messages 

External 

Internal 


We  concluded  that  message  design  addresses  only  the  presentation  part  of  an  instructional  strategy  and  is  focused  on 
specifications  for  patterns  of  signs  and  symbols  developed  by  a designer  or  learner.  This  definition  puts  the  emphasis  on 
one  part  of  the  instructional  process,  the  external  or  internal  form  of  a presentation.  This  relationship  is  illustrated  in 
Figure  1 . Notice  that  the  presentation  part  of  an  instructional  strategy  can  incorporate  many  messages. 

This  study  needs  to  be  interpreted  using  this  definition: 

Message  design  is  planning  for  the  physical  form  of  the  presentation  part  of  an  instructional  strategy  and  for  the 
symbolic  form  in  which  a message  is  to  be  stored  in  memory. 

It  was  decided  that  this  study  would  focus  on  developing  a procedure  for  designing  the  presentation  part  of  a message. 
Because  the  symbolic  form  for  storage  is  under  the  control  of  the  learner,  it  is  not  predetermined. 
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A unit  on  bicycle  safety  is  being  designed  for  second  graders  at  all  elementary  schools  in  a suburban 
middle  class  district.  The  unit  was  prompted  by  the  states  new  helmet  law  for  anyone  riding  a bicycle.  It  will 
be  team  taught  by  teachers  from  several  subject  areas  including  physical  education,  social  studies,  science. 

The  teaching  team  is  reviewing  materials  and  approaches  from  many  sources  in  order  to  determine  the  nature 
of  the  unit. 

The  objectives  for  the  unit  represent  different  types  of  goals:  informative,  attitudinal,  conceptual,  and 
psychomotor.  The  design  team  has  decided  that  students  need  (a)  knowledge,  such  as  the  safety  equipment 
required,  (b)  attitudes,  such  as  respecting  safety  laws,  (c)  concepts,  such  as  procedures  for  safe  operation  of  the 
bicycle,  and  (d)  motor  skills,  such  as  braking  and  turning. 

Thus,  unit  or  lesson  objectives  could  relate  to: 
identifying  safety  features  on  equipment 
valuing  wearing  a helmet 
obeying  laws  when  operating  a bicycle 
demonstrating  how  to  stop  a bicycle  when  riding 

Many  other  objectives  could  be  derived  from  the  goals  of  the  teaching  team.  Some  of  the  initial  elements  that 
the  team  has  decided  to  incorporate  in  the  unit  are: 

There  will  be  a mock  street  environment  created  on  the  parking  lot. 

An  existing  computerized  driver’s  test  will  be  used. 

A new  computer  program  on  the  parts  of  a bicycle  will  be  created. 

An  information  booklet  or  handout  for  parents  will  be  produced  by  second  graders.  This  booklet  will 
summarize  the  parent’s  responsibilities  and  the  child’s  responsibilities  and  may  include  a contract. 

An  assembly  program  with  a speaker's  script  and  visuals  will  be  incorporated. 

The  design  team’s  research  has  revealed  several  useful  sources  of  content,  materials  and  activities. 

The  AAA  (American  Automobile  Association)  provides  a brochure  on  bicycle  safety  laws  and  other 
materials. 

The  American  Academy  of  Pediatrics  has  an  injury  prevention  program  called  ”tipp”  that  offers  advise 
for  parents  including  guidelines  on  bicycle  use  and  maintenance.  The  "tipp”  program  also  provides  a bicycle 
driver’s  test  for  young  riders  that  parents  can  administer. 

A local  Youth  Cycle  League  offers  to  conduct  safety  awareness  activities,  such  as  assemblies, 
instructors,  and  bicycle  activity  days  with  mock  driving  tests. 

The  local  police  also  offer  these  activities. 

A national  cycling  association  offers  reprints  from  its  magazine  ’’Bicycle  Forum.”  Reprints  of 
articles  from  this  magazine  are  available  through  a local  cycling  club  called  ’’The  Wheelman.”  This  local 
club  has  a monthly  newsletter  that  stimulates  interest  in  serious  cycling,  a telephone  hotline  of  cycling  news, 
and  a web  site  for  cycling  enthusiasts. 

Design  an  instructional  message  that  will  facilitate  achievement  of  a lesson  objective.  Follow  the 
procedure  for  theory-based  message  design. 


The  Research  Process 


The  research  process  used  was  formative  evaluation  (Seels  and  Glasgow,  1990;  Flagg,  1990;  Tessmer,  1993).  The 
formative  evaluation  techniques  differed  depending  on  the  phase  of  development  and  the  stage  of  formative  evaluation.  In 
the  beginning,  focus  groups  of  advanced  instructional  design  students  explored  message  design  procedures  and 
relationships  using  the  traditional  steps  for  the  use  of  such  groups  (Stewart  & Shamdasani,  1990). 

The  first  focus  group  began  by  concentrating  on  how  to  evaluate  the  thoroughness  of  the  message  design  process. 
The  group  recommended  a checklist  based  on  a design  template  presented  in  West,  Farmer,  and  Wolff  (1991).  It  proved 
too  long  and  too  broad  to  allow  the  designers  to  focus  on  the  detail  necessary  in  message  design.  The  time  spent  on 
needs  assessment,  specification  of  objectives,  and  instructional  strategy  established  a complete  context  for  the  message  at 
the  expense  of  time  spent  on  planning  and  specifying  details.  This  original  checklist  also  incorporated  reference  to  lists 
of  principles  on  message  design  which  overwhelmed  the  designer.  Teams  within  the  focus  group  tried  out  the  procedure 
by  designing  messages  for  different  types  of  learning.  The  solutions  were  then  subjected  to  expert  review  and  revision. 
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The  next  focus  group  addressed  how  to  condense  these  variables  The  group  recommended  a one-page  template  to 
summarize  factors  such  as  context,  constraints,  learner  characteristics,  outcomes,  content  and  media.  Originally  this 
template  included  both  outcomes  and  types  of  learning.  Eventually  these  were  merged.  The  factor  of  paradigm 
considerations  was  included  on  the  template  but  was  developed  as  a separate  step  based  on  data  from  tryout.  Based  on 
.recommendations  for  revision,  the  template  became  a design  aid  but  was  eventually  disregarded  as  other  design  aids  were 
created  which  replaced  it. 

Many  worksheets  were  developed  and  discarded.  After  try-outs,  such  as  using  a long  worksheet  to  develop  a message 
to  change  attitudes,  it  became  clear  that  they  were  impractical.  Long  worksheets  overwhelmed  the  designer  particularly 
when  used  before  a few  important  decisions  were  made.  This  focus  group  also  decided  that  motivation  and  perception  were 
learning  outcomes  basic  to  all  other  types  of  learning.  Thus,  these  were  separated  from  other  types  of  learning,  and  a 
procedure  for  designing  for  these  outcomes  was  incorporated  in  the  process. 

One  of  the  last  steps  in  the  research  procedure  was  to  develop  a simulation  exercise  requiring  application  of  the 
procedure  to  solving  typical  message  design  problems.  Students  in  advanced  instructional  design  classes  used  this 
exercise  individually  and  in  teams  to  practice  the  procedure  while  being  observed  by  the  developers  who  recorded 
difficulties  with  the  procedure.  A motivational  message  on  wearing  a helmet  for  safe  cycling  was  designed  using  the 
procedure,  piloted,  and  discussed.  The  bicycle  example  which  proved  so  flexible  and  easy  was  used  with  variations  for 
exercises  and  examples  with  different  focus  groups.  An  example  of  the  bicycle  exercise  used  in  the  tryouts  is  summarized 
in  Figure  2.  Individuals  in  these  focus  groups  also  submitted  reaction  forms  and  notes  in  margins  of  the  exercise. 

As  the  procedure  was  evolving,  many  forms  for  visualizing  the  related  conceptual  framework  were  considered 
including  concentric  circles  and  a scattergram.  Eventually,  decision  trees  in  various  forms  including  hierarchies  were  used 
to  represent  both  the  conceptual  framework  and  procedure  for  message  design.  The  process  became  more  complex  than 
decision  trees  could  represent.  Therefore,  late  in  the  research  process  it  was  decided  that  an  oval  diagram  could  be  used  to 
identify  the  interactions  among  variables.  Some  conventions  from  objective  tree  diagraming  were  combined  with  oval 
diagraming  (Delp,  Thesen,  Motiwalla,  & Seshadri,1977)  to  show  how  variables  impinge  on  message  design.  From  this 
basic  visual,  the  final  conceptual  framework  emerged. 

The  Conceptual  Framework 

Because  there  is  little  theory  that  suggests  ways  to  interrelate  variables,  such  as  aesthetics,  learning  requirements  and 
situational  factors,  most  message  design  is  done  by  developers  using  creativity  and  principles  of  systems  design.  Few 
designers  are  involved  in  message  design  based  on  learning  or  other  principles.  Instructional  designers  could  prepare 
specifications  for  messages  based  on  theory  drawn  from  several  areas  of  research  if  both  conceptual  and  procedural  theory 
were  available.  A proposed  conceptual  framework  is  shown  in  Figure  3.  The  major  variables  identified  are  learning 
types,  parameters,  paradigms,  motivation,  perception,  and  design  principles.  Although  all  these  variables  are  interrelated, 
dotted  lines  are  used  to  show  those  which  have  the  closest  relationships. 

Figure  3.  A Conceptual  Framework  for  Message  Design 


See  Appendix  E of  the  1996  AECT  Proceedings 


A Theory-Based  Procedure  for  Message  Design 

This  procedure  is  intended  for  competent  instructional  designers  or  developers.  It  is  not  intended  as  a teaching  tool. 
The  results  of  formative  evaluation  indicate  that  it  is  an  effective  procedure  for  practioners  because  it  disciplines  them  to 
be  systematic  and  cues  them  to  attend  to  variables  in  specific  ways. 

Figure  4.  A Flowchart  of  the  Theory-Based  Procedure 

See  Appendix  E of  the  1996  AECT  Proceedings 
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This  procedure  requires  using  a series  of  design  aids  which  suggest  principles  that  may  be  important  to  your 
message.  None  of  these  design  aids  offers  all  the  principles  in  an  area.  You  can  also  consider  principles  you  know  are 
important  or  consult  sources.  The  design  aids  serve  to  cue  your  attention  to  areas. 

As  the  framework  shows,  the  order  in  which  you  address  the  major  variables  can  vary;  however  the  procedure  given 
recommends  an  order  that  the  research  revealed  was  generally  most  effective.  This  order  is  given  in  Figure  4.  Notice  that 
the  first  step  is  to  describe  the  purpose  for  the  message  and  the  creative  idea  under  consideration.  The  final  product  is  the 
Message  Design  Worksheet  which  summarizes  the  design  plan.  An  example  of  this  final  product  is  given  in  Table  1. 

There  are  six  design  aids  incorporated  in  the  procedure.  Each  highlights  some  of  the  principles  that  need  to  be  taken 
into  account  when  designing  for  different  types  of  learning.  Identify  the  type  of  learning  (retention/recall,  concept 
learning,  problem  solving,  attitudes,  or  motor  skills)  that  your  message  will  facilitate.  The  first  time  through  the 
procedure  it  is  best  to  work  with  only  one  type  of  learning  and  one  principle  from  each  design 
aid.  After  one  is  familiar  with  the  process,  it  becomes  easier  to  work  with  more  than  one  principle,  parameter,  or  type  of 
learning.  The  design  aid  for  types  of  learning  is  given  in  Table  2. 

The  principles  represent  the  key  questions  relevant  to  each  learning  outcome.  A designer  could  consult  more 
complete  compendiums  of  guidelines  and  principles  specific  to  these  types  of  learning. 


Table  1.  Example  of  a Completed  Message  Design  Worksheet 


Variable 

Principle 

Purpose/Idea 

.Second-grade  students  will  value  wearing  a helmet  when  riding  a bike.  A 
videotaped  presentation  comparing  the  consequences  of  accidents  with  and  without 
a helmet  will  be  produced  and  used  as  part  of  an  assembly. 

Type  of 
Learning 

Attitude  Change:  Use  sources  that  the  learner  can  identify  with  and  will 
consider  credible.  Members  of  the  local  Youth  Cycle  League  will  be  asked  to 
narrate  the  videotape  and  present  it  at  the  assembly. 

Parameters  of 
Message 

Age  and  Maturity:  Students  at  this  age  can  frighten  easily.  It  is  important  to 
emphasize  serious  consequences  without  raising  anxiety  to  a level  of 
defensiveness.  Do  this  by  offering  helmet  as  a way  to  avoid  consequences  and 
procedures  as  a way  to  avoid  accidents. 

Channel:  This  will  be  a group  presentation.  The  average  student  usually  learns 
best  from  an  effective  combination  of  audio  and  visual.  Since  this  is  attitudinal 
learning,  it  is  not  necessary  to  have  physical  practice. 

Media:  Video  sequences  using  narration. 

Content:  Use  an  analogy  of  Humpty  Dumpty  situation  or  egg  falling  if  not 
packaged.  Show  symbols  of  where  law  made  and  who  enforces  law.  Include 
fashionable  as  well  as  out-of-date  bikes  and  helmets.  Talk  about  peer  pressure. 

Learning 

Paradigm 

Cognitive  Science:  Script  for  assembly  should  include  mental  rehearsal  of 
visual  image  of  wearing  helmet  when  riding.  Guided  fantasy  or  imagine  putting 
on  your  helmet.  What  kind  of  helmet  is  it,  etc.  Include  some  non-examples  in 
video  and  ask  audience  is  he/she  ready.  Why  not?  Who’s  not  dressed  properly  for 
riding? 
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Table  1.  (continued) 


Principles  of 
Motivation 

Build  Confidence:  In  the  videotape  and  the  script  for  assembly  giv  them  a 
challenge  that  they  can  meet.  Talk  about  what  to  look  for  when  buying  a helmet, 
how  to  get  one  if  parents  can’t  afford  one  now,  and  what  to  do  until  have  helmet. 

Principles  of 
Perception 

Plan  for  Preattentive  Perception  Processing:  Start  videotape 

presentation  by  explaining  the  safe  bicycle  riding  depends  on  things  that  must 
work  together  including  the  rider  and  how  he/she  is  dressed  (with  helmet),  the  bike 
and  its  safety  equipment,  and  the  rider’s  attitude  and  knowledge. 

Design 

Principles 

Visual  Design:  Focus  attention  through  color,  e.g.  wild  helmet,  and  use  color 
in  clothing  to  create  character. 

Description  of  Design:  A videotape  presentation  for  a scripted  assembly  will  be  narrated  by  a member  of 
the  Youth  Cycle  Club.  The  assembly  will  be  hosted  by  a member  of  this  club.  The  videotape  will 
begin  by  explaining  that  safe  riding  depends  on  things  that  must  work  together  including  the  rider's 
attitude,  knowledge,  attire,  the  bike's  equipment,  and  the  parent's  rules.  Color  will  be  used  to  highlight, 
enliven,  and  make  scenes  realistic.  Analogies  of  Humpty  Dumpty  and  eggs  packaged  and  unpackaged 
will  be  given.  Pictures  of  the  state  capitol  and  police  will  be  included.  Fashionable  and  old  bikes  and 
helmets  will  be  included. 

The  topic  of  peer  pressure  will  be  discussed.  Another  topic  will  be  how  to  tell  parents  about  buying  a 
helmet  and  what  to  do  if  parents  don’t  have  the  money  now.  While  the  videotape  is  showing,  students 
will  be  asked  to  yell  out  if  characters  are  improperly  attired.  The  tape  will  end  by  offering  the  helmet  as  a 
way  to  avoid  the  consequences  of  accidents.  After  the  tape,  the  host  or  hostess  (opposite  sex  from 
narrator)  should  bring  students  up  and  put  helmets  on  some  in  line  up  and  ask  students  to  yell  out  when 
standing  behind  one  who  is  guilty  of  not  obeying  the  law.  Then,  the  students  should  be  lead  in  mentally 
rehearsing  imagining  putting  on  helmet  and  going  on  a guided  fantasy  ride. 

The  second  design  aid  summarizes  parameters  that  may  influence  the  design.  This  aid  is  given  in  Table  3.  This  aid 
cues  consideration  of  learner  characteristics,  and  channel,  media,  and  content  requirements. 

The  Learner  Characteristics  column  suggests  some  factors  that  can  affect  other  parameters.  For  example,  learning 
style  can  indicate  channel  requirements  (Smith  & Ragan,  1993),  or  ability  and  attitude  can  indicate  content  parameters. 

The  Channel  Recommendations  column  cues  the  designer  to  identify  the  means  of  sensory  input.  This  step  is 
important  because  it  limits  what  media  can  be  used  and  how  media  are  used.  A decision  to  use  the  visual  channel  only 
means  that  if  a continuous  visual  presentation  is  used,  such  as  video,  it  will  be  used  without  a soundtrack,  which  is  an 
acceptable  though  seldom  used  way  to  design  video  sequences. 

The  Media  Recommendations  column  prompts  reaffirmation  of  media  design  decisions.  This  decision  should  be 
reconsidered  in  light  of  learner,  channel,  and  content  requirements.  Integrated  technologies  includes  the  option  of  using 
people,  for  example,  to  deliver  a speech.  There  seemed  to  be  no  place  for  people  in  this  procedure.  Yet,  people  are 
demonstrably  the  best  way  to  communicate.  Because  people  usually  employ  several  channels  and  media  when  they 
deliver  a message,  this  option  has  been  placed  with  integrated  technologies. 

The  Content  Recommendations  column  reminds  you  that  messages  are  based  on  symbolic  language  in  both  visual 
and  verbal  form.  Therefore,  important  decisions  must  be  made  about  what  signs  and  symbols  to  use  and  what  examples 
and  topics  will  serve  as  vehicles.  It  is  usually  best  to  incorporate  at  least  fifty  percent  familiar  content  as  well  as  some 
novelty  for  motivation  and  challenge.  The  more  anxious  the  learner,  or  the  lower  in  ability  the  learner,  the  more 
important  it  is  to  use  familiar  imagery  and  language.  Analogies  and  metaphors  can  be  used  to  convey  meaning. 

Sometimes  there  are  reasons  why  a designer  wants  instruction  to  be  consistent  with  a learning  paradigm.  On  the 
other  hand,  the  designer  may  feel  that  in  practice,  the  best  strategy  should  be  used  regardless  of  paradigm  consistency.  In 
practice,  some  of  the  strategies,  such  as  chunking  and  case  study,  are  used  historically  by  more  than  one  paradigm.  In 
many  cases,  followers  of  one  paradigm  do  not  agree  with  the  narrowness  of  their  philosophy  as  described  by  followers  of 
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another  paradigm.  Nevertheless,  whether  one  paradigm  is  to  be  used  consistently  or  elements  of  several  paradigms 
combined  rationally,  it  is  reasonable  to  consider  how  a paradigm  applies  to  the  design  you  are  planning.  This  design  aid 
can  reveal  inconsistencies  that  should  be  eliminated.  Table  4.  is  the  Design  Aid  on  Learning  Paradigms. 


Table  2.  Design  Aid  for  Types  of  Learning 


Type  of  Learning  Principle 

Example 

Recall/Retention 

Make  associations  meaningful 

Use  meaning  as  a basis  for 
organization  or  grouping 

Repeated  exposure  to  stimuli 
enhances  retention 

Use  varied  repetition 

Provide  memory  cues 

Use  imagery  and  mnemonics 

Concept  Learning 

Carefully  select  examples 

Use  a wide  variety 

Sequence  rationally 

Use  simple  to  complex  or 
superordinate 

Confirm  concept  learning 

Have  students  identify  or  use 
concepts 

Problem  Solving 

Facilitate  acquisition  of 
information 

Emphasize  through  adjunct 
questions  and  highlighting 
techniques 

Provide  guidance  in  structuring 
the  problem 

Demonstrate  translating  the 
problem  into  sentences  or 
drawings 

Facilitate  learning  how  to 
separate  relevant  and  irrelevant 
information 

Provide  practice  in  selecting 
information  from  cues  and 
directions 

Attitude  Change 

Influence  persuasiveness 
through  choice  of  sources 

Use  sources  that  the  learner  can 
identify  with  and  with  enough 
information,  experience,  and 
intelligence  to  be  considered 
credible 

Establish  belief  congruence 

Argue  in  favor  of  a position 
the  learner  holds  so  that 
effectiveness  is  increased  when 
presenting  an  argument  in 
another  area 

Provide  balanced  arguments  that 
relate  to  the  learner's  needs 

Create  and  manage  dissonance; 
show  how  needs  can  be 
satisfied  by  adoption;  recognize 
current  attitudes  and  move  in 
small  steps  towards  the  desired 
attitude;  state  the  conclusion 
explicitly 
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Motor  Skills 

Cue  mental  practice 

Cue  steps  in  procedure 

Match  presentation  format  to 
type  of  task 

Use  observation  of  model 
performance  for  simple  tasks 
incorporating  known  skills 

Sequence  appropriately  for  task 
characteristics 

Use  the  whole-task  approach 
for  integrated  and  coordinated 
activities 

Because  inoti  ation  is  basic  to  all  other  types  of  learning,  every  instructional  message  should  incorporate  principles 
of  motivation.  These  design  aids  summarize  the  most  important  questions  to  consider.  They  are  meant  to  be  used  with 
more  extensive  references  in  the  literature  on  principles  of  motivation  and  on  tactics  for  implementing  these  principles. 
For  example,  McAleese  and  Gunn  (1994)  list  tactics  for  developing  confidence  (expectancy)  as  providing  tests  of 
performance,  feedback,  praise,  advisement,  and  knowledge  of  results.  Table  5 is  the  aid  for  motivation. 


Table  3.  Design  Aid  for  Parameters  of  Message 


Learner 

Characteristics 

Channel 

Recommendations 

Media 

Recommendations 

Content 

Recommendations 

Demographic,  e.g. 
age,  maturity,  socio- 
economic background, 
geographic  region, 
etc. 

Learning  style,  e.g. 

visual/auditory, 

sensory /intuitive, 

inductive/deductive 

active/reflective, 

sequential/global 

Ability  and  attitude 
e.g.  high  or  low, 
anxious,  enthusiastic 

Audio 

Visual 

Audiovisual 

Kinesthetic 

Tactile 

Olfactory 

Print  technologies, 
e.g.  frame,  page 

Audiovisual 
technologies,  e.g. 
shot,  sequence 

Computer-based 
technologies,  e.g. 
graphic,  screen, 
sequence 

Integrated 
technologies,  e.g. 
sequence,  people 

Sources  of  imagery 
e.g.  cultural, 
professional 

Signs 

Symbols 

Examples 

Topics 
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Table  4.  Design  Aid  for  Paradigms 


Behaviorism 

Cognitive  Science 

Constructivism 

Paradigm 

Assumptions 

performance  oriented 

domain  oriented 

problem  context 
oriented 

know  reality 

construct  schemata 

construct  reality 

designer  perspective 

designer  and  learner 
perspective 

multiple  perspectives 

mathemagenic 

strategies 

mathemagenic 

strategies 

generative  strategies 

acquire  knowledge 

socially  constructed 
meaning 

Micro  Strategies 

contiguity 

reinforcement 

cuing 

feedback 

minimize  errors 

chunking 

frames 

mapping 

advance  organizer 

rehearsal 

mnemonics 

outlining 

metaphor/analogy 

visuals 

argument 

discussion 

debate 

reflection 

exploration 

interpretation 

construction 

use  misconception 

Symbols 

direct  attention 

cue  memory 

tools  for  constructing 
reality 

external  representation 
of  reality 

external  and  internal 
representation  of 
reality 

internal  representation 
of  reality 

If  learners  do  not  attend  the  message,  then  communication  and  learning  will  not  occur.  Like  motivation,  perception 
is  basic  to  all  types  of  learning.  Use  the  Design  Aid  on  Perception  Principles  to  ensure  that  the  audience  receives  what  is 
intended.  It  is  assumed  that  the  designer  could  at  this  point  consult  more  complete  compendiums  of  principles  specific  to 
types  of  learning,  such  as  perception.  Table  6 is  the  aid  for  perception. 

There  are  instructional  and  aesthetic  principles  of  design  that  come  from  media  areas  such  as  graphic  design,  visual 
communication,  information  design,  screen  design,  and  text  design.  These  principles  can  be  used  to  solve  problems  that 
are  not  unique  to  a type  of  learning,  such  as  how  to  use  space  and  how  to  organize  symbols.  Only  a few  of  the  many 
sources  are  suggested  on  the  design  aid.  Use  the  principles  offered  to  critically  evaluate  the  design  for  the  message;  then, 
revise  the  design  as  suggested  by  the  principles.  Start  by  identifying  a column  (screen,  text,  information,  or  visual 
design)  that  relates  to  the  message  you  are  developing.  Table  7 presents  the  aid  for  general  design  principles. 

The  last  step  in  the  procedure  is  to  summarize  a design  plan  based  on  the  principles  identified  on  the  Message  Design 
Worksheet.  The  form  for  this  worksheet  is  presented  in  Table  1 which  was  presented  earlier  in  this  article.  After 
summarizing  the  principles  and  parameters  selected,  a paragraph  is  written  describing  the  creative  approach  to  design  and 
the  theory  exemplified  by  the  design. 
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Table  5.  Design  Aid  for  Principles  of  Motivation 


Type  of  Learning 

Principle 

Example 

Motivation 

Stimulate  learner's  attention 
and  curiosity 

Use  variation  in  organization 
and  presentation;  provoke 
mental  conflict 

j 

Create  a clear  relationship 
between  objectives  and  learner 
goals 

Use  analogies,  metaphors,  and 
personal  language  to  build  a 
relationship  between  content 
and  objectives  and  learners’ 
need  and  desires 

Build  confidence  to  produce 
appropriate  expectancy  for 
success 

Design  appropriate  challenge 
level 

Gain  and  maintain  learner's 
attention 

Make  the  presentation 
appealing  by  using  print, 
writing  style,  graphics, 
formatting  and  images 

O 
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Table  6.  Design  Aid  for  Principles  of  Perception 


Type  of  Learning 

Principle 

Example 

Perception 

Plan  for  preattentive  perceptual 
processing 

Organize  information  using 
clear  boundaries,  clear  figure- 
ground  distinctions  and  clear 
units  or  parts 

Select  and  organize  information 
to  assist  in  attentive  perceptual 
processing 

Sequence  information 
appropriately  (top-bottom,  left- 
right  for  English  readers) 

Plan  for  learner  interpretation 

Consider  learner  characteristics, 
such  as  cognitive  abilities,  e.g. 
skills 

Structure  visual  perception 

Provide  good  figures  which  are 
complete,  balanced,  and 
perceived  as  integrated  rather 
then  separate  elements;  group 
by  proximity,  similarity, 
common  movement 

This  design  plan  should  be  reviewed  by  colleagues  and  representative  learners  and  adjusted  as  recommended.  After 
completing  this  theory-based  procedure,  the  next  phase,  developing  or  producing  the  message  starts.  Changes  in  the 
message  design  concept  can  occur  at  this  next  stage  also. 

Discussion 

Performance  support  systems  take  many  forms  at  different  levels  of  sophistication  including  calculators,  word 
processors,  workstations  and  just-in-time  training  on  demand.  This  conceptual  framework  and  procedure  was  conceived  as 
a basis  for  interactive  on-the-job  training  technology  that  would  support  message  design  activities.  The  database 
necessary  for  message  design  is  so  extensive  that  an  "if-then"  performance  support  system  would  be  the  ideal.  This 
electronic  performance  support  system  could  have  as  one  of  its  components  a decision  making  model  based  on  this 
theory-based  procedure. 

This  performance  support  system  could  also  include  an  expert  system  to  present  an  ideal  approach  to  decision 
making,  text  retrieval  to  make  references  conveniently  available  and  searchable,  a data  base  to  provide  access  to  principles 
and  guidelines,  a browser  to  extend  access  to  sources  of  information,  and  a neural  network  system  to  connect  key  words 
in  the  knowledge  domain.  Obviously,  this  proposed  theory  does  not  provide  the  interface  or  elements  to  develop  such  a 
system.  Nevertheless,  it  is  a start  towards  conceptualizing  a performance  support  system  that  could  accept  input  and 
adapt,  provide  multiple  modes  of  access  to  information,  and  offer  both  practice  activities  and  activities  which  require 
organizing  knowledge.  Task  support  worksheets  could  be  supplemented  by  cue  cards,  wizards,  coaching,  explanations, 
checklists,  tips,  decision  trees,  decision  making  models,  and  process  maps.  The  user  could  study  through  computer- 
based  instructional  sequences,  look  up  information  through  on-line  reference  systems  in  hypertext,  and  ask  advise  of  an 
intelligent  expert  system.  Roles  for  experts  and  novices  could  be  programed  including  roles  for  designers,  writers, 
producers,  artists,  programers,  and  managers. 

What  form  will  the  adaptation  of  technology  for  message  design  functions  take?  Will  expert  systems  lead  designers 
through  the  process  or  replace  the  designer?  In  the  future  will  message  design  be  done  by  machines  or  a man-machine 
system?  Properly  designed  systems  could  allow  non-experts  to  produce  results  similar  to  an  expert.  This  would  be  done 
by  the  expert  system  querying  the  user  to  gain  data  about  the  situation.  This  data  would  then  be  processed  by  the  system 
following  built-in  rules. 
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The  process  of  message  design  becomes  far  less  daunting  when  a database  holds  all  the  worksheets  and  checklists  to 
be  used  Once  part  of  that  database,  the  items  that  apply  to  the  current  design  can  be  selected  and  created  as  working 
documents  through  screens  committing  the  result  to  paper.  Performing  any  job  well  calls  for  the  consistent  application 
of  a set  of  rules  or  procedures.  We  all  omit  a step  from  time  to  time,  but  the  computer  never  skips  a step.  Once 
properly  programed  the  computer  is  capable  of  replicating  results  time  after  time.  A debatable  issue,  however,  is  whether 
such  a computerized  support  system  would  eliminate  the  creativity  that  distinguishes  many  superior  messages. 

One  advantage  of  such  systems  could  be  the  ability  to  manage  the  minutia  of  many  design  projects.  Many  designers 
are  imaginative  but  uninterested  in  details.  Turning  the  details  of  a project  over  to  an  electronic  assistant  frees  the 
designer  to  be  creative  without  losing  control  of  the  details.  The  computer  could  ensure  full  coverage  of  all  the  necessary 
steps  in  the  process  of  message  design  presented  by  this  conceptual  structure. 
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Abstract 


Pre-enrollment  factors,  particularly  those  related  to  computer  confidence  in  using  computer  applications,  were 
investigated  in  relation  to  persistence  in  the  community  college.  Data  on  sex,  ethnicity,  reading  level,  high  school  grade 
point  average  (GPA),  goal  for  attending  college,  and  self-reported  ability  to  use  five  computer  application  programs: 
word  processing,  database,  spreadsheets,  graphics,  and  on-line  communications  programs  were  gathered  from  543 
traditional,  entering  students  and  analyzed  in  relation  to  the  students’  persistence  at  the  beginning  of  the  subsequent 
semester.  Univariate  and  multivariate  statistical  analyses  were  employed.  Results  of  the  logistic  regression  indicate  that 
ethnicity,  goal  for  attending  college,  confidence  in  using  spreadsheets  for  college  work,  and  the  interaction  between  high 
school  GPA  and  total  number  of  computer  skills  that  the  student  is  confident  using  for  college  work  are  related  to 
persistence  in  college.  While  future  studies  are  needed  to  determine  direct  causal  effects,  community  college 
administrators  may  wish  to  consider  designing  pilot  programs  in  computer  skills  training  for  the  at  risk  entering  student. 


Confidence  in  Types  of  Computer  Use:  Factors  Associated  with  Retention  in  the  Community  College 

Thirty-nine  percent  of  the  students  pursuing  higher  education  in  the  nation  are  enrolled  in  two  year  colleges  (U. 

S.  Department  of  Education,  1994).  Although  enrollment  levels  for  the  two  year  college  population  are  projected  to 
increase,  a critical  concern  among  administrators  at  these  institutions  is  student  retention.  National  survey  data  on  the 
attrition  of  two-year  college  students  from  1972  through  1990  show  that,  based  on  full-time,  or  a combination  of  full- 
and  part-time,  enrollment  statistics,  40%  to  44%  of  entering  students  do  not  persist  to  the  second  year  (Tinto,  1993). 
Although  some  of  these  academic  leavers  may  not  consider  themselves  “dropouts”  but  rather  “stopouts”  who  plan  to 
return  to  college  at  some  point  in  time  (Bonham  and  Luckie,  1993;  Conklin,  1993),  their  behaviors  can  be  costly  to  their 
institutions,  particularly  as  institutional  resources  become  more  limited.  Colleges  can  plan  resource  allocations  more 
effectively  with  consistent  patterns  of  enrollment;  students  can  also  plan  their  futures  more  effectively  when  following  a 
consistent  attendance  pattern. 

At  the  same  time,  the  nature  of  the  K- 12  experience  for  recent  high  school  graduates  has  been  changing  as  their 
potential  for  being  exposed  to  newer  technologies  has  increased.  Data  on  78%  of  16,100  public  school  districts  and 
intermediate  units  in  the  United  States  reflect  a change  from  a ratio  of  125  students  per  computer  since  the  1983-1984 
school  year  to  a ratio  of  14  students  per  computer  in  1993-1994,  along  with  increased  use  of  modems,  local  area 
networks  and  peripherals  (Quality  Education  Data,  1994b).  Projected  spending  for  computer  hardware  in  1994  was 
estimated  at  $10,202,527,565  (Quality  Education  Data,  1994a). 

The  question  arises  of  whether  this  increase  in  computing  resources  in  the  schools  shows  indications  of 
contributing  to  success  in  college.  More  specifically,  Qj:  Do  traditional,  entering  community  college  students  who 
indicate  confidence  in  their  ability  to  use  one  or  more  computer  application  program(s),  or  in  their  ability  to  use  a 
specific  type  of  computer  application  program  (word  processing,  graphics,  database,  spreadsheets,  on-line 
communications)  to  carry  out  college  assignments  demonstrate  greater  persistence  in  college  after  one  semester  than  those 
who  do  not  indicate  such  ability?  Additionally,  Q2:  Are  the  variables  of  gender,  reading  level,  high  school  grade  point 
average  (GPA),  ethnicity,  and  goals  factors  in  this  relationship? 

Background 

Student  pre-enrollment  factors,  such  as,  gender,  ethnicity,  high  school  GPA,  college  aspirations;  student  college 
factors,  such  as,  college  GPA,  academic  and  social  integration  into  college,  full-  and  part-time  enrollment  status;  and 
institutional  characteristics,  such  as,  contact  with  faculty,  access  to  resources,  and  availability  of  financial  aid,  have  all 
been  identified  as  playing  a complex  role  in  student  retention  (Gaither,  1992;  Pascarella  & Terenzini,  1991 ; Tinto,  1994). 
The  complex  role  that  confidence  plays  toward  applying  computer  skills  to  academic  work  has  also  emerged  in  regard  to 
factors,  such  as,  age  and  motivation  (Klein,  Knupfer,  & Crooks,  1993),  gender,  computer  experience,  and  age  (Loyd  & 
Gressard,  1984),  computer  literacy  levels  (Hignite  & Echternacht,  1992;  Mahmood  & Medewitz,  1989),  and  prior 
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knowledge  of,  use  of,  and  access  to  computers  (Malaney,  & Thurman,  1989).  Despite  extensive  studies  on  college 
retention  and  on  computer  literacy,  little  research  has  been  designed  to  explore  relationships  between  the  two. 

Broadly  defined,  computer  literacy  includes  student  self-efficacy  toward  computer  use  as  well  as  knowledge  about 
computers  and  ability  in  computer  usage  (Simonson,  Maurer,  Montag-Torardi  & Whitaker,  1987).  In  their  meta- 
analysis of  twenty  years  of  college  retention  research,  Pascarella  and  Terenzini  (1991)  state  that  ‘‘...the  impact  of  college 
is  largely  determined  by  the  individual’s  quality  of  effort  and  level  of  involvement  in  both  academic  and  nonacademic 
activities”.  Might  one,  therefore,  ask  if  levels  of  computer  literacy  could  be  related  to  some  degree  to  the  “quality  of 
effort”  that  a student  expends  in  college? 

While  concerns  have  also  arisen  over  providing  students  with  up-to-date  computers  and  equitable  access  to 
computers,  more  importantly,  research  is  needed  related  to  computer  use  and  learning  (Apple,  1991;  Becker,  1991; 
Billings,  1986;  Muffoletto  & Knupfer,  1993).  Given  the  direct  high  cost  of  obtaining  and  maintaining  computers  and 
the  indirect  cost  of  training  teachers,  as  well  as  the  impact  of  computers  on  curriculum  (Billings,  1986),  there  is  a 
continuing  need  for  research  that  examines  the  relation  of  computers  to  various  aspects  of  learning. 

The  purpose  of  this  study  was  to  investigate  a broad  learning  outcome,  persistence  in  college,  in  relation  to 
entering  community  college  students’  attitudes  toward  computers  . Focus  was  placed  on  the  relationship  of  confidence  in 
the  computer  use  of  commonly  used  computer  application  programs  to  persistence  in  college  in  an  attempt  to  gather 
preliminary  data  on  the  pre-enrollment  characteristic  of  computer  confidence  in  relation  to  these  specific  computer 
applications. 


Method 

The  study  was  designed  as  a preliminary  investigation  into  the  relationship  between  persistence  in  community 
college  and  computer  ability. 

The  participants  were  traditional,  first  year  entering  students  who  participated  in  the  regular  Orientation,  Test, 
Advising,  and  Registration  sessions  (OTAR)  conducted  throughout  the  summer  of  1993  at  one  of  the  largest  community 
colleges  in  the  University  of  Hawaii  System.  Recent  high  school  graduates  were  selected  because  of  the  likelihood  that 
they  would  have  had  experience  using  computers  and  would  have  formed  opinions  about  their  ability  to  use  computer 
applications  for  carrying  out  college  assignments. 

As  part  of  the  OTAR  process,  the  students  complete  a questionnaire  designed  to  gather  personal  background  and 
altitudinal  data.  For  this  study  survey  items,  constructed  by  the  authors,  were  added  to  gather  data  on  the  confidence 
students  had  in  using  specific  computer  application  programs  to  carry  out  college  work.  This  specialized  strand  of  the 
overall  analysis  of  the  assessment  process,  focused  on  determining  if  such  computer  usage,  specifically,  word  processing, 
computer  graphics,  spreadsheets,  databases,  and  ability  to  access  information  electronically  on-line  could  be  factors  in 
subsequent  retention  in  college.  The  students’  responses  concerning  confidence  using  computer  applications  were  also 
calculated  to  obtain  a total  score,  hereafter  referred  to  as  computer  score.  That  score  had  the  potential  of  ranging  from 
zero  to  five. 

On  the  computer  survey,  students  were  presented  with  a forced  choice  option  of  “yes”,  ”no”,  and  “uncertain”  as  it 
was  unrealistic  to  ask  them  to  indicate  the  degree  to  which  they  were  confident  in  using  these  application  programs  to 
carry  out  college  assignments.  Either  they  would  use  the  application  to  carry  out  a college  assignment  or  they  would 
not.  The  uncertain  category  was  included  to  discourage  some  students  from  skipping  a question.  This  study  was 
designed  to  collapse  the  uncertain  responses  into  the  “no”  response  section  during  the  data  analysis  phase  of  the  study. 
Pilot  testing  of  the  instrument  carried  out  prior  to  the  initiation  of  the  regular  test  sessions,  employing  a test-retest 
approach  and  interviewing  techniques  resulted  in  obtaining  a reliability  coefficient  of  .98,  indicating  high  reliability  for 
the  instrument. 

Six  hundred  fifteen  students  responded  to  the  computer  survey  and  to  the  request  for  information  on  pre- 
enrollment characteristics  of  gender,  ethnicity,  goal  for  attending  college,  entry  level  academic  ability  reflected  through 
high  school  GPA,  and  reading  score  on  the  Nelson  Denny  reading  test. 
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At  the  start  of  the  second  semester  these  students  were  tracked  to  check  for  re- enrollment  in  the  college.  Using 
student  identification  numbers,  students  were  separated  into  groups  delineated  as  persisters  and  as  non-persisters.  From 
the  initial  615  participants,  a total  of  543  students  consisting  of  those  who  had  persisted  and  those  who  had  left  the 
college,  were  identified  as  having  complete  data  sets  n = 418  and  n = 125  respectively.  No  attempt  was  made  in  this 
study  to  distinguish  between  “dropouts”  and  “stopouts”. 

Data  Analysis 

Data  obtained  from  the  participants  were  coded  and  entered  into  a computer  database  for  analysis  using  SPSS  for 
UNIX.  Univariate  comparisons  were  employed  to  test  for  differences  between  the  two  sets  of  students,  persisters  and 
non-persisters,  in  relation  to  the  selected  pre-enrollments  factors.  The  chi-square  statistic  was  used  to  test  for  the 
significance  of  the  relationship  between  persistence  in  college  and  these  factors. 

Cells  where  small  numbers  were  reported  were  collapsed,  resulting  in  the  ethnic  categories  of  “Blacks”  and 
“Hispanics”  being  added  to  the  “Other”  ethnic  category.  For  the  same  reason,  during  the  multivariate  analysis  process, 
computer  scores  of  three  through  five  were  collapsed  into  a new  category  entitled,  “three  or  more”. 

Treating  the  computer  score  and  high  school  GPA  as  continuous  variables,  the  possible  relationship  of 
confidence  in  ability  to  use  one  or  more  computer  applications  in  combination  with  high  school  GPA  levels  to 
persistence  in  college  was  tested  through  a one-way  analysis  of  variance. 

To  investigate  the  complex  relationship  of  the  factors  together  in  relation  to  the  dichotomous  criterion  variables 
of  persistence  and  non-persistence  in  college,  logistic  regression  was  employed  using  a forward  stepwise  approach  that 
selects  predictors  in  order  of  importance.  The  forward  stepwise  method  was  chosen  because  of  its  applicability  to  college 
retention  research  studies  designed  to  measure  the  dichotomous  dependent  variables  of  persistence  and  non-persistence  in 
retention  (Dey  & Astin,  1993)  and  because  of  its  power  in  selecting  variables  to  remain  in  the  final  model  (Retherford  & 
Choe,  1993).  The  significance  of  each  variable  was  tested  with  the  Wald  statistic.  The  probability  for  entering  a 
variable  into  the  model  was  set  for  p < .05  and  the  probability  for  removing  it  set  at  < 0.1. 

Results 

The  results  of  the  chi-square  analyses  for  the  pre-entry  level  characteristics  of  gender,  ethnicity,  entry  level 
academic  ability,  reading  ability,  and  goal  for  attending  college  indicate  that  gender,  high  school  GPA,  and  reading  level 
as  measured  by  the  Nelson  Denny  reading  test,  are  not  significantly  related  to  retention.  Ethnicity  and  goal  for  attending 

2 2 

college  were  shown  to  be  significantly  related  to  retention  in  college,  c (4,  N = 543)  = 14.58,  p < .01  and  c (,  N = 

543)  = 1 1 .46  p < .01  respectively.  Asians  were  identified  as  the  ethnic  group  that  persisted  at  a greater  rate  than 
anticipated.  Filipinos  persisted  slightly  above  the  anticipated  rate.  Persistence  rates  were  less  than  anticipated  for  all 
other  ethnic  groups. 

Those  students  whose  goal  was  transferring  to  a four  year  college  and  those  students  who  indicated  their  goal 
was  different  from  the  standard  options  for  attending  community  college  persisted  at  higher  rates.  Interpretations  about 
the  significance  of  the  latter  group  should  be  made  cautiously,  however,  because  of  the  small  number  of  respondents  in 
this  category  who  persisted  (n  = 1 8). 

Being  a confident  user  of  computer  applications  or  not,  was  not  significantly  related  to  persistence  in  college. 
Those  who  persisted  were  less  likely  to  express  confidence  for  using  word  processing,  graphics,  databases,  and  on-line 
communications.  Those  students  who  were  confident  about  their  ability  to  use  spreadsheet  applications  were  the  only 
type  of  specific  computer  application  users  who  showed  some  indication  of  being  more  likely  to  persist  than  anticipated. 

An  examination  the  results  presented  in  the  crosstabulation  tables  in  relation  to  computer  score,  showed  no 
clear  pattern  of  use  emerging  in  regard  to  the  number  of  skills  students  indicated  they  were  confident  in  applying.  The 
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analysis  of  the  interaction  of  the  two  variables:  high  school  GPA  and  computer  score  indicated  a significant  effect , j>  = 

.01. 


Table  1 lists  the  results  of  the  forward  stepwise  logistic  regression  in  which  the  important  variables  were 
examined  together,  presenting  variables  that  remained  in  the  equation  along  with  their  coefficient,  standard  error  and 
indication  of  significance  levels.  The  standards  set  for  the  equation  were:  for  gender,  being  male;  for  ethnicity,  being 
Asian;  for  high  school  GPA,  having  an  average  GPA;  for  reading,  reading  at  or  above  the  standard;  for  goal,  planning  to 
transfer  to  a four  year  college;  and  for  each  individual  computer  application,  indicating  being  a confident  user.  For  the 
computer  score  the  standard  was  set  at  zero.  The  same  standards  were  retained  for  the  interaction  variable  of  high  school 
GPA  and  computer  score. 

The  results  shown  in  the  display  of  the  logistic  regression  model  indicate  that  ethnicity,  goal  for  attending 
college,  confidence  in  using  spreadsheet  computer  applications,  and  the  interaction  between  high  school  GPA  and 
confidence  in  total  computer  skills  are  significantly  related  to  persistence  in  college. 

Of  particular  note  is  that  the  majority  of  the  significant  variables  indicate  a negative  effect  on  persistence.  For 
example,  the  results  suggest  that  those  students  who  identify  their  ethnicity  as  being  Hawaiian  or  Pacific  Islander,  White 
or  Other,  are  significantly  less  likely  to  persist  in  college  than  the  ethnic  group  used  as  the  standard,  the  Asians.  This 
negative  effect  is  true  for  those  students  whose  goal  for  college  is  personal  enrichment  and  obtaining  a two  year  degree. 
They  are  less  likely  to  remain  in  college  than  their  comparison  group,  those  whose  goal  is  to  transfer  to  a four  year 
school 


The  computer  related  variables  that  reflect  significance  are  positively  related  to  retention.  Students  who  express 
confidence  toward  using  spreadsheet  applications  for  college  work  are  significantly  more  apt  to  persist  than  those  who  do 
not  indicate  such  confidence.  In  comparison  to  students  with  average  high  school  GPA’s  who  have  indicated  confidence 
in  using  no  computer  applications,  students  with  low  high  school  GPA’s  and  three  or  more  computer  skills  and  those 
with  high  GP/  ’s  and  two  computer  skills  are  significantly  more  likely  to  remain  in  college.  As  both  these  groups  have 
small  numbers  of  students,  with  the  latter  group  containing  less  than  fifteen  students,  the  results  of  this  interaction 
should  be  interpreted  cautiously. 

The  factors  of  gender,  high  school  GPA  alone,  reading  level,  and  confidence  in  the  separate  skills  of  using  word 
processing,  databases,  graphics,  on-line  communications  and  computer  score  alone  did  not  contribute  to  the  model.  - 

The  model  created  by  the  results  of  the  logistic  regression  correctly  classified  76.8%  of  the  cases  in  terms  of 
persistence  in  college;  26.8%  more  than  would  be  accounted  for  by  chance.  Its  limitations  arise  in  its  ability  to  predict 
persisters  more  accurately  than  it  predicts  non-persisters. 

Discussion 

The  results  of  this  study  indicate  that  certain  pre-entry  student  variables  do  relate  to  persistence  in  college  with 
the  categories  of  organismic  factors,  goal  for  attending  college,  and  confidence  in  employing  computer  applications  to 
college  work  each  contributing  to  the  resultant  model.  The  investigation  answered  the  research  question,  Qj,  with  the 
finding  that,  for  students  in  this  study,  being  confident  in  applying  a specific  computer  application,  spreadsheets,  is 
indicated  as  being  the  only  computer  application  that  appears  related  to  persistence  in  college  in  conjunction  with  the 
other  variables  selected  in  the  model.  Being  confident  in  using  a total  number  of  computer  applications  appears  to  only 
be  related  to  persistence  for  students  with  either  low  or  high  GPA’s.  Closer  scrutiny,  however  is  warranted  because  of 
the  small  number  of  cases  that  contributed  to  the  effect. 

While  at  first  it  may  appear  surprising  that  having  confidence  in  using  spreadsheets  was  the  one  specific 
computer  application  that  contributed  to  the  equation,  it  may  be  a result  of  its  possible  relationship  to  mathematical 
ability,  an  aspect  for  which  spreadsheet  applications  are  predominantly  used,  and  an  aspect  for  which  this  study  was  not 
designed  to  investigate.  Future  studies  may  show  that  the  spreadsheet  factor  is  an  effect  of  mathematical  ability. 
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Research  question  02  was  addressed  with  the  finding  that  ethnicity,  goal  for  attending  college  and  high  school 
GPA  in  interaction  with  computer  score  appears  to  influence  persistence  in  college  for  the  students  in  this  study. 


The  confidence  in  spreadsheet  use  factor  was  selected  for  inclusion  in  the  equation  during  the  multivariate 
analysis  of  the  data,  although  it  was  not  a significant  factor  when  considered  by  itself.  The  lack  of  contribution  that  the 
other  specific  computer  applications  made  to  the  model  is  not  surprising,  given  the  complexity  of  determining  the 
relationship  between  attitudes  toward  computers  and  computer  literacy.  Hignite  and  Echtemacht  (1992)  have  documented 
this  condition  in  studies  with  college  populations.  Mixed  results  may  be  found  regarding  the  relationship  between 
students*  ability  to  use  computers  and  planned  future  use.  Larson  and  Smith  (1993)  found  that  entering  college  students 
who  were  computer  users  were  less  likely  to  have  positive  attitudes  toward  using  computers.  For  the  students  in 
Malaney*s  and  Thurman’s  (1989)  college-based  study,  however,  computer  use  before  entering  college,  when  combined 
with  access  to  computers  and  anticipated  future  computer  use,  showed  indication  of  serving  to  identify  college  users. 

Although  both  the  Larson  and  Smith  (1993)  sample  and  the  participants  in  this  study  both  gave  indication  of 
being  able  to  use  word  processing,  spreadsheets  and  graphics  applications,  more  than  half  of  the  former  indicated 
capability  with  these  three  applications;  for  the  students  in  this  study  such  confidence  in  using  applications  were  indicated 
only  for  word  processing.  The  lower  confidence  levels  evidenced  by  these  students  may  reflect  the  more  demanding 
wording  of  the  items  in  the  survey  where  respondents  were  asked  to  state  their  confidence  in  using  the  various  computer 
applications  at  the  level  of  “carrying  out  college  assignments**.  Computer  usage  in  the  Larson  and  Smith  survey  simply 
inquired  about  “familiarity**  with  the  applications. 

The  omission  of  the  confidence  in  word  processing  factor  from  the  final  logistic  regression  equation  appears  to 
be  related  to  findings  by  Oliver  and  Kerr  (1993)  and  Power,  Fowles,  Famum,  and  Ramsey  (1992)  whose  studies  of  actual 
word  processing  use  by  college  students  indicate  that  its  use  does  not  contribute  to  improved  grades  and  scores.  In  regard 
to  college  students*  writing,  Oliver  and  Kerr  found  that  the  level  of  revisions,  rather  than  the  use  of  word  processing 
contributed  to  improved  grades  on  college  students*  essays.  Powers  et  al.  found  that  handwritten  versus  word  processed 
work  resulted  in  higher  scores  for  the  handwritten  submissions. 

As  this  study  sought  only  to  investigate  the  effect  of  pre-enrollment  factors,  the  small  contribution  that  this 
resulting  model  for  persistence  in  college  makes  in  identifying  non-persisters  may  be  explained  by  the  work  of  Gates  & 
Creamer  (1984),  who  found  in  their  review  of  a sampling  of  community  college  student  records  from  a national  database, 
that  pre-enrollment  variables  account  for  only  4.3%  of  the  variation  in  retention  when  such  variables  are  the  only  type  of 
variables  used  in  the  analysis.  This  study  was  not  designed  to  gather  data  on  student  college  or  institutional  factors  that 
might  contribute  to  persistence  in  college.  Future  studies  could  be  designed  to  investigate  the  effect  that  the  addition  of 
such  data  would  have  on  the  resulting  model. 


Summary  and  Conclusions 

This  study  sought  to  answer  questions  concerning  the  percentage  of  entering  community  college  students  who 
are  computer  confident  and  whether  this  confidence  has  any  immediate  impact  on  retention.  If  the  cutting  edge  campus  of 
the  future  is  to  be  on  the  information  superhighway,  how  important  is  it  for  students  to  arrive  at  the  campus  compute, 
confident?  Although  limited  by  being  based  on  the  results  of  participants  from  one  community  college  whose  student 
body  has  a higher  proportion  of  Asian,  Filipino,  and  Hawaiian  or  Pacific  Islanders  than  other  community  colleges;  it 
does  provide  preliminary  data  and  a framework  in  which  other  researchers  may  attempt  to  answer  these  and  similar 
questions  as  they  begin  to  address  the  broad  questions  posed  by  Bork  (1993)  with  regard  to  the  need  for  determining 
effective  uses  for  computers.  While  future  studies  are  needed  to  determine  direct  causal  effects,  the  results  of  this  study 
raise  questions  about  the  type  and  time  frame  for  providing  technological  support  to  entering  community  college 
students.  Community  college  administrators  may  want  to  consider  developing  pilot  computer  skills  training  sessions  to 
be  offered  prior  to  college  entry  for  the  traditional,  entering  high  risk  students  to  determine  the  possible  contribution  that 
such  training  may  have  on  subsequent  student  persistence  in  college. 
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Table  1 

Logistic  Regression  on  the  Effect  of  Selected  Pre-Enrollment  Characteristics  on 
Persistence  in  the  Community 


Independent  Variables 

S.E. 

Coefficient 

Logit 

Organism ic  Variables 
Ethnicity 

Hawaiian/Pacific  Islander 

.3592 

-1.2071*** 

White 

.3894 

- .9655** 

Filipino 

.3631 

- .3652 

Other,  Black,  Hispanic,  American  Indian 

.3675 

-1.0334** 

Academic  Characteristics  and  Goals 
Goal  for  attending  college 

Personal  enrichment 

.8008 

-1.6382* 

Vocational/Technical  certificate 

.5314 

- .7373 

Two  year  degree 

.3136 

- .8513** 

Other 

.6491 

.3660 

Confidence  in  Computer  Skills 

Spreadsheet 

.3110 

.8767* 

Interactions  with  Confidence  in  Computer  Skills 
GPA  by  confidence  in  total  computer  skills 

Low  GPA  by  one  computer  skill 

.7123 

.8060 

Low  GPA  by  two  computer  skills 

.8260 

- .4212 

Low  GPA  by  three  or  more  computer  skills 

.8837 

3.2919*** 

High  GPA  by  one  computer  skill 

.6882 

.5669 

High  GPA  by  two  computer  skills 

.8487 

1.6734* 

High  GPA  by  three  or  more  computer  skills 

.6538 

.8493 

Intercept 

.2470 

.6697** 

Note.  Reference  categories  for  the  categorical  variables  in  order  of  appearance  above  are  Asian,  transfer  to  a four  year 
college,  average  GPA,  and  confidence  in  spreadsheet  skills.  For  the  interaction  terms  the  reference  is  average  GPA  and 
confidence  in  zero  computer  skills.  Variables  that  were  not  selected  during  the  forward  stepwise  variable  selection  are 
gender,  high  school  GPA,  reading  level,  confidence  in:  word  processing,  database,  graphics,  on-line  communications 
skills,  and  confidence  in  total  number  of  computer  skills. 

*p<  .05;  **p<  .01,  ***p<001 
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Abstract 


Keller  (1983)  has  described  motivation  as  “the  heart”  of  our  understanding  of  how  to  design  effective  instruction. 
Stimulating  interest  and  reducing  boredom  are  important  goals  for  promoting  learning  achievement.  This  article 
explores  boredom  and  interest  in  educational  settings,  examines  their  relationship  to  the  characteristics  of  emotion, 
and  describes  the  results  of  a research  study  intended  to  identify  motivating  factors  and  their  sources  in  education. 

Introduction 

Much  of  the  literature  on  instruction  and  learning  focuses  on  cognitive  outcomes  that  support  one  theory  or  another. 
Even  though  a learner’s  attitudes  and  motives  can  affect  the  results  of  even  the  most  carefully  planned  and  executed 
instructional  intervention  and  continuing  motivation  has  been  called  the  major  educational  outcome  of  significance” 
(Sorensen  & Maehr,  1976),  educational  researchers  seldom  examine  the  motivational  antecedents  and  outcomes  of 
learning  (Engelhard  and  Monsaas,  1988).  As  a result,  we  know  much  more  about  the  cognitive  elements  of 
instruction  than  about  learners’  feelings  of  involvement  and  enjoyment  (Dodge,  1980). 

Motivation  is  what  Keller  (1983)  calls  “the  heart”  of  our  understanding  of  how  to  design  effective  instruction.  White 
(1959)  wrote  of  an  innate  need  to  feel  effective,  while  Deci  (1975)  described  motivation  as  a need  to  feel  competent 
and  self-determining.  Both  of  these  needs  were  used  to  explain  why  we  do  things  that  have  no  subsequent  extrinsic 
rewards,  why  we  are  interested  in  some  activities  for  their  own  sake.  Furthermore,  emotional  factors  (combined  with 
motivation,  ability,  and  personality  factors)  have  been  found  to  contribute  to  learning  and  performance  (Boyle, 

1983).  One  possible  explanation  for  the  lack  of  substantive  research  is  the  multifaceted  nature  of  learner  motivation. 

This  article  (1)  examines  the  research  on  interest  and  boredom,  (2)  explores  interest  and  boredom  as  emotional  states, 
(3)  describes  some  motivational  elements  and  sources  of  boring  and  interesting  learning  situations  through  analysis 
of  learners’  descriptions,  and  (4)  examines  the  attributions  for  those  descriptions. 

Interest  and  Boredom 

Interest  encompasses  curiosity,  engagement,  and  arousal  (Berlyne,  1963).  Sorensen  & Maehr  (1976)  describe  interest 
as  the  basis  for  the  continuing  motivation  to  learn.  Keller  (1983)  links  interest  to  the  arousal  and  sustaining  of  a 
learner’s  curiosity.  Malone  (1981)  asserts  that  intrinsically  motivating  environments  depend  on  how  well  they 
continue  to  arouse  and  satisfy  our  curiosity. 

Keller’s  ARCS  Model  of  Motivational  Design  (e.g.  1987)  designates  four  factors  (Attention,  Relevance,  Confidence, 
Satisfaction)  as  critical  preconditions  to  a learner’s  motivation  to  learn.  Keller  defines  attention  as  a broad  term 
encompassing  interest  and  curiosity.  The  ARCS  Model  has  as  its  foundation  expectancy-value  theory — that 
motivation  is  predicated  on  the  importance  (value)  of  the  task  and  the  perceived  ability  to  achieve  it  (expectancy  for 
success).  There  is  evidence  that  the  Attention  and  Relevance  factors  contribute  to  the  valuing  of  a learning  task  while 
the  Confidence  and  Satisfaction  factors  contribute  to  the  learner’s  expectation  for  learning  success  (Small  & Gluck, 
1994).  The  ARCS  Model  is  the  only  instructional  design  model  to  examine  learner  interest  and  prescribe  a range  of 
instructional  strategies  that  promote  it. 

Duffy  (1972)  found  links  between  reported  interest  and  value.  He  found  that  not  only  did  adolescents  claim  greater 
interest  in  what  they  found  valuable,  but  they  also  asked  more  questions  and  learned  and  remembered  more.  Amone 
& Small  (1995)  theorize  that  without  learner  attention  and  interest,  there  can  be  no  relevance,  confidence,  and 
satisfaction  potential.  They  reason  that  a lack  of  learner  interest  results  in  failure  to  pay  attention  to  instruction  and, 
therefore,  none  of  the  other  motivations  could  emerge.  They  perceive  learner  interest,  specifically  curiosity,  as  a 
foundation  of  motivation. 

Damad-Frye  & Laird  (1989)  concede  that  as  an  emotion,  boredom  shares  many  of  the  same  processes  as  other 
emotions  (e.g.  happiness,  anger).  Berlyne  (1960)  and  others  (e.g.  Fiske  & Maddi,  1961;  Geiwitz,  1966)  agree  with 
Damad-Frye  & Laird’s  contention  that  the  basic  element  of  boredom  behavior  is  the  “struggle  to  maintain  attention” 
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(p.  316)  and  define  boredom  as  “a  metacognitive  judgement  about  one’s  attentional  activity”  (p.  320).  When  learners 
are  bored,  they  pay  less  attention,  resulting  in  lower  retention  and  less  ability  to  apply  information. 

Berlyne  (1960)  describes  boredom  as  a state  that  occurs  when  external  stimuli  are  excessively  scarce  or  monotonous, 
or  what  Keller  (1987)  might  describe  as  “a  lack  of  perceptual  arousal.”  Robinson  (1975)  found  that  monotony  and 
repetition  were  perceived  major  causes  of  boredom.  Bernstein  (1975)  provides  one  view  of  boredom  as  “a  feeling,  an 
affective  response  to  an  appropriate  external  situation”  (p.  513).  Fenichel  (1951)  claims  that  boredom  arises  when 
we  must  not  do  what  we  want  to  do  or  must  do  what  we  do  not  want  to  do”  (p.  359).  Marsh  (1983)  found  that 
students  engage  in  certain  behavioral  patterns  simply  to  mask  their  boredom  and  attempt  to  appear  interested. 
Boredom  may  also  be  a result  of  information  underload— too  little  information  that  is  interesting  or  information 
overload — too  much  information  that  is  not  interesting  (Klapp,  1973). 

However,  boredom,  like  interest,  is  a complex  feeling  state  in  which  inattention  may  only  be  one  dimension. 

Perkins  & Hill  (1985)  found  that  although  boredom  was  not  significantly  correlated  with  “meaningless  stimulation,” 
the  term  “meaning”  may  have  been  interpreted  differently  by  different  subjects;  i.e.,  a stimulus  might  be 
“meaningful”  if  it  is  perceived  as  relevant  to  the  perceiver’s  motive  satisfaction  or  frustration.  Dodge’s  (1981) 
“Boredom  Analysis  Flowchart”  provides  some  links  between  boredom  and  instruction  that  is  too  complex,  too 
difficult,  and/or  too  detailed.  The  result  may  be  lack  of  confidence  and  feelings  of  incompetence.  Farmer  & Sundberg 
(1986)  found,  among  other  things,  a negative  relationship  between  boredom  and  life  satisfaction.  All  of  these  infer  a 
relationship  between  interest  (or  attention)  and  the  other  ARCS  components. 

Both  pleasantness  and  self-determinism  may  also  be  conditions  of  interest,  while  lack  thereof  may  point  to 
conditions  of  boredom  (Kopp,  1982).  Kopp’s  first  step  for  designing  boredom  out  of  instruction  is  to  analyze  the 
potential  for  arousal,  pleasantness,  and  self-determinism.  He  recommends  instructional  strategies  such  as  novelty, 
surprise,  incongruity,  and  uncertainty  as  anti-boredom  strategies. 


The  Study  of  Emotion 

For  much  of  this  century,  psychologists  tended  to  avoid  the  empirical  study  of  emotion.  There  were  three  reasons  for 
this  avoidance:  1)  behavioral  scientists  tended  to  view  emotion  as  a unitary,  global  concept,  which  made 
operationalization  difficult;  2)  stimulus-response  drive-reduction  principles  dominated  psychology;  and  3)  there  was 
no  adequate  theory  dealing  with  separate  and  distinct  emotions,  each  definable  as  a construct  that  could  be  studied  by 
specified  and  repeatable  operations.  As  a result,  the  realm  of  emotion  remained  a relatively  unexplored  territory  (Izard, 
1971). 

When  emotion  was  discussed  within  theories  of  motivation  and  behavior,  it  was  often  regarded  as  unidimensional. 
Hedonistic  theories  of  motivation  tended  to  view  emotion  along  a single  continuum  of  pleasantness-unpleasantness 
(e.g.,  McClelland,  Atkinson,  Clark,  and  Lowell,  1953).  Activation  theorists  redefined  emotion  as  arousal,  varying 
along  a sleep-tension  dimension.  One  even  advocated  abandoning  the  term  emotion  altogether  (Duffy,  1962). 

One  aspect  of  emotion  that  lent  itself  to  scientific  study  was  the  area  of  facial  expressions.  Typical  experiments 
involved  the  rating  of  paired  photographs  of  an  actor  displaying  a range  of  emotions.  In  those  studies,  two 
dimensions  appeared:  Pleasant-Unpleasant,  and  Sleep-Tension  (Schlosberg,  1954;  Gladstones,  1962).  A third 
dimension  sometimes  appeared  in  such  studies,  but  it  was  usually  weak  and  difficult  to  inteipret.  Abelson  and 
Sermat  (1962)  concluded  that  a two-dimensional  model  adequately  accounted  for  differences  in  facial  expression. 

In  contrast  to  studies  of  facial  expression,  research  on  verbal  reports  of  emotion  has  often  shown  a three-dimensional 
structure.  Izard  (1972),  for  example,  gave  subjects  the  names  of  the  eight  fundamental  emotions  (fear,  shyness, 
interest,  distress,  anger,  guilt,  joy,  and  surprise)  in  his  typology  and  asked  them  to  recall  a situation  in  their  lives  in 
which  each  emotion  was  strongly  experienced.  For  each  situation,  subjects  also  filled  out  a rating  scale  which  asked 
them  how  active,  deliberate,  tense,  impulsive,  controlled,  self-assured,  extraverted,  and  pleasant  they  felt. 

Analysis  of  the  ratings  showed  that  the  Pleasantness,  Tension,  and  Self-Assurance  dimensions  were  the  best 
combination  to  distinguish  the  eight  emotional  situations.  Joy,  for  instance,  was  characterized  as  being  high  in 
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Pleasure  and  Self-Assurance,  and  low  in  Tension.  Izard  concluded  that  the  three  dimensions  of  Pleasure,  Tension,  and 
Self-Assurance  represent  the  underlying  structure  of  subjective  experience  of  which  emotion  is  one  aspect. 


<7 


In  another  attempt  to  uncover  the  dimensions  of  emotion.  Bush  (1973)  asked  subjects  to  rate  pairs  of  emotional 
adjectives  (e.g.,  sleepy,  outraged,  delighted)  on  a 10-point  similarity  scale.  The  ratings  were  analyzed  by 
multidimensional  scaling  analysis.  Three  dimensions  emerged:  the  first  two  were  Pleasantness-Unpleasantness  and 
Level  of  Activation.  These  are  similar,  if  not  identical  to  the  Evaluation  and  Activity  dimensions  found  in  semantic 
differential  research,  (Osgood,  Suci,  & Tannenbaum,  1957),  as  well  as  to  factors  found  in  studies  of  facial 
expression. 


Bush  (1973)  labeled  the  third  factor  found  in  his  study  “Aggression,”  though  he  noted  that  the  scale  was  not  easily 
interpreted.  At  one  end  were  adjectives  like  “outraged”  and  “delighted,”  while  at  the  other  were  “sympathetic,” 
“needed”  and  “desperate.”  He  concluded  that  this  dimension  was  closely  related  to  the  Potency  scale  of  the  semantic 
differential  and,  more  specifically,  seemed  to  refer  to  interpersonal  potency. 


Additional  support  for  a three-factor  model  of  emotion  comes  from  work  by  Mehrabian  and  Russell  (1974)  in  ttfe 
field  of  environmental  psychology.  Based  on  previous  research,  they  theorized  that  the  emotional  response  to  an 
environment  can  be  described  along  the  dimensions  of  Pleasure,  Arousal,  and  Dominance,  in  parallel  to  Evaluation, 
Activity,  and  Potency  (Osgood,  Suci,  & Tannenbaum,  1957).  They  constructed  an  18-item  measure  based  on  these 
factors  and  used  it  both  as  a measure  of  personality  and  as  a state  measure  of  emotional  response.  As  they  predicted, 
the  measure  was  found  to  relate  to  measures  of  anxiety  (high  Arousal,  low  Pleasure  and  Dominance),  sensitivity  to 
rejection  (low  Dominance),  and  several  other  measures.  A state  emotional  response  scale  (Mehrabian  & Russell, 
1974)  was  used  to  measure  reactions  to  a number  of  environmental  situations,  and  the  three  dimensions  of  emotion 
were  significant  predictors  of  various  approach  and  avoidance  behaviors. 


A later  study  (Russell  & Mehrabian,  1977)  used  the  same  scale  to  rate  adjectives  which  depicted  a full  spectrjim  of 
emotions.  A total  of  151  terms  were  used,  and  each  subject  rated  10  to  20  of  them.  Again,  the  Pleasure,  Arojsal,  and 
Dominance  scales  showed  high  reliability  (.97,  .89,  and  .87)  and  all  three  scales  were  necessary  to  distinguish  among 
the  emotional  adjectives.  * 

Interest  and  Emotion  \ 

With  the  exception  of  work  by  Csikszentmihalyi  (e.g.,  1975),  there  have  been  few  attempts  to  examine  in  lepth 
what  interest  and  involvement/ee/  like  to  an  individual.  1 


Intrinsic  motivation  presents  a fascinating  case  of  the  state  of  the  art.  Theories  and  data  abound.  Are  we 
making  major  advances  in  understanding  motivation?  Let  us  suggest  that  they  are  minor.  We  continually 
overlook  our  major  source  of  knowledge — a personal,  non-objective  source  which  is  at  the  heart  of  every 
minitheory  but  not  acknowledged.  Our  methodologies  fall  short  because  they  lead  us  into  more  and  r|ore 
detailed  specification  of  external  conditions  for  producing  behavioral  effects  and  ignore  the  critical  variable, 
namely,  the  way  the  person  experiences  (not  perceives)  the  conditions  that  we  so  elaborately  contrivd  ( 
DeCharms  & Muir,  1978,  p.  107) 

% 


This  is  not  to  say  that  emotion  has  been  totally  ignored  by  those  doing  research  on  learner  interest.  Several^ 
theoreticians  have  called  upon  emotions  as  explanatory  variables  in  their  writings.  Berlyne  (1960),  for  example,  said 
that  curiosity  comes  as  a result  of  a desire  to  maintain  an  optimal  level  of  arousal.  Things  that  are  puzzlin  Jpr 
incongruous  cause  a rise  above  this  optimal  level,  and  the  individual  studies  and  processes  the  stimulus  in  Iraer to 
reduce  uncertainty  and  bring  arousal  back  to  a more  comfortable  tonus  level.  I 

Day  (1982)  extended  Berlyne’s  work,  describing  a “zone  of  curiosity”  as  a state  of  activation  (arousal)  in  wf|ch  a 
person  is  interested  and  excited,  as  demonstrated  by  exploration  and  approaching  behaviors.  When  not  in  this  zone, 
the  person  is  either  in  a “zone  of  relaxation”  (boredom)  or  “zone  of  anxiety,”  (avoidance  behaviors  and  disiijfcrest). 1 
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Another  line  of  research  on  interest  has  involved  the  development  and  use  of  interest  inventories:  scales  to  measure 
preferences  among  sets  of  school  subjects  or  vocations.  The  interest  inventory  can  be  viewed  as  a technological 
extension  of  hedonistic  theories  of  motivation.  That  is,  they  assume  implicitly  that  we  are  pleasure-seeking  creatures 
who  arrange  our  activities  to  maximize  pleasure  (Travers,  1978). 

The  major  work  to  date  which  studied  the  actual  experience  of  interest  and  involvement  has  been  conducted  by 
Csikszentmihalyi  (e.g.  1975;  1980).  Csikszentmihalyi  surveyed  and  interviewed  chess  players,  rock  climbers, 
surgeons  and  others  to  examine  what  made  their  activities  self-rewarding.  He  discovered  several  qualities  that  seemed 
to  characterize  the  occurrence  of  intrinsic  motivation,  a state  of  mind  that  he  called  “flow.”  Among  these  were  a 
merging  of  action  and  awareness,  the  centering  of  attention,  and  the  loss  of  self-consciousness.  In  emotional  terms, 
the  flow  experience  combined  enjoyment,  a feeling  of  mastery  and  control,  and  a level  of  arousal  midway  between 
boredom  and  anxiety. 

Within  this  sample  of  inquiry  in  the  area  of  interest,  several  different  emotional  variables  have  been  cited  as  causes  or 
correlates  of  interest.  One  might  infer  from  this  that  interest  is  not  a simple,  pure  emotion  like  happiness,  for 
example,  but  a complex  combination  of  feelings.  If  this  is  so,  then  perhaps  the  existing  mini-theories  of  curiosity 
and  intrinsic  motivation  are  accurate  but  incomplete,  like  the  fabled  blind  men’s  description  of  the  elephant.  The 
tendency  to  regard  interest  as  one-dimensional  has  held  back  progress  toward  understanding  it,  and  prevented  the 
building  of  a cumulative  body  of  research.  We  posit,  then,  that  learner  interest  may  indeed  be  a multidimensional 
construct. 

A Model  of  Learner  Interest 

If  interest  is  multidimensional,  what  are  its  dimensions?  A logical  analysis  of  the  concept  would  suggest  that  the 
three  factors  underlying  other  emotions  also  make  sense  as  dimensions  of  interest.  The  Pleasure-Displeasure 
dimension,  for  instance,  is  clearly  relevant.  To  be  interested  is  a pleasant  feeling,  to  be  bored  is  not. 

The  Arousal  dimension  seems  applicable  to  interest  as  well.  Any  definition  of  interest  would  have  to  include  notions 
of  a heightened  awareness  and  level  of  attention,  both  of  which  are  associated  with  moderate  to  high  levels  of 
arousal.  Drowsiness  and  lack  of  interest,  on  the  other  hand,  are  states  of  low  arousal.  As  already  noted,  arousal  also 
figures  prominently  in  previous  research.  Perhaps  the  best  articulated  theory  of  curiosity,  that  of  Berlyne  (1960, 
1963),  is  built  around  the  concept  of  arousal.  Furthermore,  both  Day  and  Berlyne  theorize  that  only  aroused  persons 
learn  and,  when  aroused,  they  learn  better.  In  addition,  the  “flow”  state  studied. by  Csikszentmihalyi  (1975)  was 
characterized  by  a facilitative  level  of  arousal. 

Thus,  an  argument  for  Pleasure  and  Arousal  as  factors  underlying  learner  interest  is  justified.  But  what  about  the 
third  dimension,  one  that  is  related  to  the  Potency  ( ong-weak)  dimension  of  semantic  differential  research?  This 
question  might  be  rephrased  for  our  purposes  as:  In  an  educational  setting,  what  does  it  mean  to  feel  stronger  or 
weaker? 

Learning  adds  to  one’s  skills  and  abilities  and  thus  enables  one  to  deal  more  effectively  with  the  world.  In  a sense,  an 
increase  in  competence  is  an  increase  in  power.  To  feel  more  competent  is  to  feel  stronger,  while  incompetence  and 
weakness  are  inextricably  linked.  Thus  it  would  seem  that  a dimension  of  Competence-Incompetence  feelings  is  an 
appropriate  translation  of  Potency  within  the  context  of  learning. 

The  Competence  Feeling  dimension  is  tied  to  interest  by  some  of  the  theoretical  work  already  cited.  Learners  need  to 
actively  explore  situations  where  they  feel  competent  and  self-determining  (D  ci,  1975;  Keller,  1983;  Lepper,  1985). 
Deci’s  (1975)  cognitive  evaluation  theory  proposes  that  activities  which  enhance  a feeling  of  competence  and  self- 
determination  are  intrinsically  motivating.  Learners  want  to  feel  personally  effective  in  managing  their  learning 
environment  and  seek  out  intrinsically  motivating  activities  (Kinzie,  1990);  i.e.  activities  done  for  their  own  sake, 
without  the  need  for  external  compensation. 

In  an  essay  on  interest  and  effort  in  education,  Dewey  (1913)  made  a distinction  relevant  to  the  Competence  Feeling 
dimension.  He  said  that  there  are  two  types  of  pleasure.  One  arises  from  contact  with  pleasurable  stimuli  such  as 


bright  colors  and  agreeable  sounds.  The  second  accompanies  activity  and  is  found  wherever  there  is  successful 
achievement  and  mastery  occurring.  These  two  types  of  pleasure  are  reflected  in  the  present  model  as  the  Pleasure  and 
Competence  Feeling  dimensions,  respectively. 

Dewey  had  few  kind  words  for  school  activities  which  excite  the  senses  but  do  not  engage  the  learner  in  activity 
geared  toward  mastery.  This  same  concern  is  echoed  today  when  some  argue  that  instruction,  which  produces  high 
levels  of  Arousal  and  Pleasure  but  does  not  arouse  Competence  Feelings,  can  be  said  to  be  more  entertaining  than 
educational.  High  levels  of  Pleasure,  Arousal,  and  Competence  Feelings,  on  the  other  hand,  would  indicate  genuinely 
engrossing  instruction  that  promotes  learning  outcomes. 

Some  empirical  evidence  for  the  appropriateness  of  a Competence  Feelings  dimension  comes  from  a study  of  what 
makes  an  educational  television  program  interesting  (MacLean  et  al,  1960).  In  an  analysis  of  viewer  ratings  of 
several  program  segments,  factors  of  Evaluation  (good-bad)  and  Activity  (fast-slow)  emerged  which  are  closely  linked 
to  the  Pleasure  and  Arousal  dimensions  discussed  above.  In  addition,  a third  factor  appeared  which  had  to  do  with 
how  well  the  program  was  understood.  The  researchers  named  this  factor  Simplicity,  but  it  is  clear  from  the  items 
which  loaded  on  the  factor  that  it  could  as  easily  have  been  interpreted  as  a dimension  of  Competence  Feelings. 

Thus  far,  we  have  described  learner  interest  as  having  underlying  dimensions  of  Pleasure,  Arousal,  and  Competence 
Feelings.  There  is  one  more  dimension  yet  to  be  described,  one  which  also  has  to  do  with  Potency. 

Strength  and  weakness  in  an  educational  context  can  also  be  conceptualized  in  another  way.  One  is  strong  when  one 
controls  the  content  and  mode  of  presentation  of  what  is  being  learned.  Weakness  is  having  no  such  control.  This 
second  type  of  Potency  dimension  is  linked  to  learner  interest  by  the  widely  held  assumption  that  learner  control 
enhances  motivation.  This  feeling  of  being  in  charge  of  one’s  learning  is  closely  akin  to  the  Origin-Pawn  theory  of 
deCharms  (1968,  1976).  To  use  Deci’s  (1975)  terminology,  this  dimension  might  be  referred  to  as  one’s  Self- 
Determination  Feelings. 

Kinzie  (1990)  states  that  “when  actively  exercised  in  a responsive  environment,  options  for  learner  control  can 
promote  perceptions  of  personal  control  and  further  strengthen  continuing  motivation.”  (p.  8)  She  sees  learner 
control  as  contributing  to  learner  interest  and  engagement.  Kinzie  further  asserts  a direct  relationship  between 
competence  and  selfdetermination  and  links  them  to  control,  describing  them  as  factors  that  influence  the  learner’s 
interest  in  the  instruction  and  contribute  to  the  learner’s  continuing  motivation. 

Thus,  a four-factor  model  of  learner  interest  is  proposed,  consisting  of  the  three-factor  structure  of  emotion  and  a 
logical  analysis  of  the  concept  of  learner  interest.  These  four  factors  related  to  the  Evaluation,  Activity,  and  Potency 
dimensions  of  semantic  differential  research  (Osgood,  Suci,  & Tannenbaum,  1957)  appear  in  Table  1 . 


Semantic  Differential  Dimensions  Learner  Interest/Emotional  Factors 


Evaluation Pleasure 

Activity Arousal 

Potency Competence/Dominance  & Self-Determination 


Table  1.  Emotional  Factors  Related  to  the  Semantic  Differential  Dimensions 
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Uses  of  the  Model 


A model  of  learner  interest  may  be  useful  both  as  a conceptual  tool  and  as  a guide  toward  the  solution  of 
instructional  problems.  At  the  conceptual  level,  the  model  helps  to  bring  diverse  theories  and  approaches  together 
within  one  framework.  This  provides  a more  holistic  view  of  interest,  affect  and  motivation  and  makes  it  easier  to 
compare  theories  or  to  combine  different  theoretical  perspectives. 

This  model  also  enables  one  to  make  finer  distinctions  than  is  possible  with  a single  dimension  ranging  from 
boredom  to  high  interest.  The  four  factors  can  economically  portray  a wide  range  of  affective/motivational  responses 
to  instruction.  For  example,  one  form  of  boredom  might  be  represented  by  a combination  of  negative  pleasure  and 
low  arousal.  A closely  related  state  would  be  drowsiness,  which  is  mildly  pleasant  and  low  in  arousal.  The 
combination  of  high  pleasure,  moderate  arousal  and  neutral  competence  and  self-determination  feelings  might 
constitute  a sort  of  passive  fascination.  Perhaps  the  optimum  response  for  designers  to  strive  for  would  be  what 
Csikszentmihalyi  (1975,  1990)  calls  “flow,”  a state  of  total  involvement  consisting  of  moderate  arousal  and  high 
pleasure,  competence,  and  self-determination  feelings. 

It  should  be  pointed  out  that  the  emotions  associated  with  interest  and  boredom  are  less  powerfully  felt  than  are  other 
well  researched  emotions  like  fear.  Affective  reactions  to  instruction  are  subtle.  Lipson  (1979)  calls  them  “aesthetic 
emotions”  and  notes  the  total  lack  of  research  on  them. 

In  summary,  research  on  motivation  in  learning  situations  has  rarely  looked  at: 

• student  perceptions  of  what  they  find  motivating. 

• the  various  dimensions  of  interesting  and  boring  instruction. 

• the  feelings  and  emotions  learners  experience  in  boring  and  interesting  learning  situations. 

• the  instructional  causes  of  interest  and  boredom  and  their  attributed  sources. 

The  research  questions  in  this  study  were: 

1 . What  specific  instructional  factors  contribute  to  learner  boredom  and  interest? 

2.  To  what  underlying  instructional  source(s)  do  learners  attribute  those  factors? 

3.  What  are  the  dimensions  of  learning  interest  and  boredom  as  they  relate  to  emotion? 

Methods 

This  research  was  conducted  using  512  student  subjects  from  two  universities.  Two  methods  of  data  collection  were 
used: 

1 . Descriptive  responses.  Subjects  were  asked  to  reflect  on  past  learning  experiences  in  two  ways: 

• Brainstorming  sessions  were  conducted  with  350  undergraduate  and  graduate  students  at  a large  northeastern 
university.  Subjects  were  asked  to  voluntarily  describe  orally  one  boring  and  one  interesting  past  learning 
experiences  and  the  contributing  factors.  Responses  ranged  from  single  words  (e.g.  “dynamic,” 
“disorganized”)  to  phrases  (e.g.  “excited  about  subject  matter,”  “content  irrelevant”),  to  complete  sentences 
(e.g.  ‘The  instructor  used  a variety  of  teaching  methods.”  “There  were  no  real-life  applications.”) 

• Written  questionnaires  were  administered  to  162  graduate  students  at  a large  western  university.  Subjects 
were  asked  to  briefly  describe  one  boring  and  one  interesting  learning  experience  and  the  critical  factors 
contributing  to  each.  Examples  of  each  appear  below. 
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Boring:  ‘The  subject  of  the  lecture  was  “the  relationship  between  ideas  and  other  aspects  of  culture  and 
social  structure”  in  a class  titled  “Industrial  Sociology.”  The  classroom  was  fairly  full,  maybe  25-30 
people.  No-one  seemed  particularly  interested.  The  whole  lecture  seemed  to  me  to  be  a history  of  religion 
more  than  anything  else  and  I could  not  understand  its  relevance  to  what  we  were  to  learn.  He  did  not  seem 
to  care  about  the  fact  that  no-one  was  interested  in  what  he  was  saying.  He  never  gave  us  a chance  to  ask 
questions,  and  he  just  talked  on  and  on.  He  seemed  to  be  totally  wrapped  up  in  himself  and  the  fact  that  he 
knew  so  much  about  the  subject.  He  did  not  take  the  particular  audience  into  consideration.” 

Interesting:  “The  most  interested  I have  been  in  any  one  classroom  situation  was  in  a graduate  level 
statistics  class  on  probability.  It  was  a large  class  of  about  50  students.  The  instructor  did  not  use  a text 
but  supplemented  his  lectures  with  handouts  and  tasks  to  be  completed  each  week.  This  particular  class  I 
recall  not  understanding  well  before  class  but  the  instructor  was  able  to  give  examples  and  used  a story 
about  tea  leaves  so  well  that  I knew  I understood  by  the  end  of  the  3 hours.  I appreciated  the  teacher’s  style 
and  ability  to  explain  a difficult  subject  area  in  a clear  and  interesting  way. 

2.  Subjective  evaluations.  Subjects  who  completed  the  open-ended,  written  questionnaire  also  completed  two 

Likert-type  scales. 

• Scale  1 . This  scale  required  subjects  to  rate  their  boring  and  interesting  learning  situations  on  a seven-point 
scale  containing  nine  bipolar  terms  derived  from  the  literature.  These  terms  were  predictable-unpredictable , 
complex-simple , impersonal-personal,  easy-impossible,  colorless-colorful,  changing- unchanging, 
ineffective- effective,  familiar- unfamiliar,  irrelevant  to  my  needs- relevant  to  my  needs. 

• Scale  2.  This  scale  required  subjects  to  rate  their  emotions  during  each  of  the  two  learning  situations  they 
described  on  a seven-point  scale  containing  48  pairs  of  unrelated  terms.  The  67  terms  reflected  a wide  range 
of  emotional  terms  found  in  the  literature  that  related  to  the  emotional  factors  (i.e.,  pleasure,  arousal, 
competence,  self  determination).  The  unrelated  terms  were  randomly  assigned  to  each  scale  item;  as  a 
result,  23  terms  appeared  more  than  once.  Three  terms  appeared  three  times  (dull,  excited , snobbish)  and 
one  term  (cruel)  appeared  four  times.  Some  examples  of  paired  terms  appearing  on  this  scale  are  cruel - 
happy,  excited-untroubled , dull-humiliated,  in  control-cared  for , sad-f rustrated. 


Results 

Descriptive  response  data  were  used  to  answer  both  Research  Question  #1  (What  specific  instructional  factors 
contribute  to  learner  boredom  and  interest?)  and  Research  Question  #2  (To  what  underlying  instructional  source(s)  do 
learners  attribute  those  factors?). 

Brainstorming  responses  were  transcribed  from  overhead  transparencies.  Each  response  was  reviewed  and,  when 
multiple  ideas  were  represented,  broken  down  into  separate  single  ideas  for  analysis.  For  example,  one  subject 
volunteered  the  following  complex  response  which  was  broken  down  into  four  separate  ideas: 

• His  lecture  consisted  of  telling  us  exactly  what  we  had  read  the  night  before. 

• He  never  translated  the  difficult  textbook  we  had. 

• He  only  made  things  foggier. 

• He  talked  slowly  and  in  a monotone. 

All  responses  were  analyzed  by  a panel  of  three  motivation  experts  who  were  trained  in  the  ARCS  Model.  Each 
expert  categorized  the  responses  in  terms  of  Keller’s  ARCS  Model  motivation  components  [Attention  (A), 
Relevance  (R),  Confidence  (C),  Satisfaction  (S)]  and  attribution;  i.e.,  to  what/whom  they  attributed  responsibility 
for  the  situation  [Instructor  (I),  Learners  (L),  Materials  (M),  Environment  (E)]  Categorization  continued  until 
consensus  was  reached.  In  cases  where  no  consensus  could  be  reached,  the  items  were  eliminated  from  the  analysis. 
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A total  of  780  responses  were  analyzed  (see  Tables  2 and  3).  There  were  slightly  more  interesting  statements  (407; 
52%)  than  boring  statements  (373;  48%).  The  panel  was  unable  to  categorize  57  statements  according  to  ARCS 
categories  (e.g.  ‘The  instructor  was  handsome”).  All  statements  were  successfully  categorized  by  attributed  source. 
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Table  2.  Interesting  Statements 
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Table  3.  Boring  Statements 


Results  of  this  analysis  related  to  Research  Question  #1  (What  specific  instructional  factors  contribute  to  learner 
boredom  and  interest?)  indicated: 

• Attention  statements  were  the  most  common  for  both  boring  (63%)  and  interesting  (51%)  situations.  As  such, 
Attention  may  be  the  most  influential  for  generating  interest.  Furthermore,  attention  and  relevance  accounted 
for  the  vast  majority  of  boring  (84%)  and  interesting  (75%)  situations.  This  may  indicate  that  the  value  of  the 
learning  task  is  the  predominant  influence  on  both  positive  and  negative  motivations. 

• In  boring  situations,  instructional  materials  were  perceived  as  most  important  for  providing  relevance  (63%), 
while  the  instructor  was  most  important  for  providing  attention  (58%),  confidence  (63%),  and  satisfaction 
(70%). 

A 

Results  related  to  Research  Question  #2  (To  what  underlying  instructional  source(s)  do  learners  attribute  those 
factors?)  indicated: 

• The  instructor  was  the  overall  source  of  more  than  half  of  the  statements  in  both  situations  (boring:  52%; 
interesting:  58%);  even  higher  for  Attention  alone  (boring:  58%;  interesting:  66%).  However,  although 
subjects  attribute  the  majority  of  the  responsibility  for  their  motivation  to  the  instructor’s  performance,  other 
factors  also  play  a role. 
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• Subjects  considered  themselves  (the  learners)  a more  influential  factor  in  interesting  situations  (24%)  than  in 
boring  ones  (6%).  That  is,  it  appears  that  subjects  believed  that  when  a learning  event  is  interesting,  it  is 
more  often  due  to  something  they  have  done;  but  when  it  is  boring,  they  assume  almost  no  responsibility. 

• Instructional  materials  were  a more  important  factor  in  boring  situations  (38%)  than  in  interesting  (18%) 
situations.  Materials  that  are  not  perceived  as  valuable  or  contributing  to  learning  success  may  not  be  useful 
as  motivational  techniques. 

• Environmental  conditions  (e.g.  a cold  classroom,  uncomfortable  seats)  have  almost  no  influence  (boring:  4%; 
interesting:  0%)  on  the  motivational  level  of  a learning  event . 

Additional  data  related  to  Research  Question  #1  were  collected  from  162  subjects  at  one  university.  Subjects  used  a 
seven-point  scale  of  nine  bipolar  terms  (derived  from  the  literature)  to  characterize  both  their  interesting  and  boring 
learning  situations.  Values  were  assigned  so  that  each  item  on  the  scale  ranged  from  “most”  to  “least.”  For  example, 
a score  of  7 on  predictable  was  interpreted  as  “the  most  predictable”  while  a score  of  1 meant  it  was  considered  “the 
least  predictable.” 

For  the  purpose  of  this  analysis,  all  data  were  combined  and  interest  was  used  as  a dependent  variable.  Multiple 
regression  analyses  were  performed  (see  Table  4).  Among  the  nine  independent  variables,  colorful,  effective , and 
personal  emerged  as  the  best  antecedents  of  an  interesting  situation.  Predictable  appears  to  be  a complicated 
influence.  Although  predictable  had  some  negative  influence,  it  was  not  strong.  One  interpretation  might  be  that  in 
one  sense  too  much  predictability  might  cause  boredom  but  some  predictability  may  be  a positive  influence  on 
interest.  For  example,  organized  instruction  that  is  predictable  (e.g.,  structured,  organized)  may  be  positive  while 
use  of  the  same  instructional  methods  (e.g.,  lecture,  role  playing)  all  the  time  might  be  considered  a negative  type  of 
predictability.  Other  independent  variables  had  no  significant  influence  upon  the  situation’s  degree  of  interest  or 
boredom. 
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.000 
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No 

Familiar 
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No 
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.044 

1.608 

.109 

No 

Table  4.  Results  of  Multiple  Regression  Analyses  for  Nine  Independent  Variables 


A series  of  correlations  analyses  were  conducted  to  identify  the  causal  relationships  between  interest  and  each  of  the 
independent  variables.  The  results  appear  in  Table  5.  Results  indicate  that  colorful  is  the  best  antecedent  of  an 
interesting  situation  while  predictable  appears  to  be  the  best  antecedent  of  a boring  situation. 
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Colorful 

.980 

.913 
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Effective 
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.835 
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.248 
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Easy 
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.109 

.140 
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-.089 

-.117 

Predictable 

-.671 

-.549 

-.461 

-.602 

Table  5.  Correlation  Analyses  for  Interest  and  the  Independent  Variables 

Additional  results  indicate: 

• Colorful,  effective , and  personal  have  a significant  positive  influence  upon  the  degree  of  interest.  These  three 
terms  might  be  seen  as  correlates  of  the  three  dimensions  of  our  interest  model;  i.e.  colorful -arousal,  effective- 
pleasure,  personal-competence. 

• Changing  and  relevant , although  not  statistically  significant,  show  a positive  influence  upon  the  degree  of 
interest,  while  predictable  has  a significant  negative  influence  on  the  degree  of  interest.  Students  appear  to 
prefer  instruction  that  is  novel,  surprising,  varied — major  characteristics  of  Keller’s  (1987)  Attention  factor. 

• Complex,  easy , and  familiar  have  no  significant  influence  upon  the  degree  of  interest.  However,  mean  scores 
for  both  boring  and  interesting  situations  were  almost  the  same  for  complex-simple  (boring:  3.0;  interesting: 
3.3)  and  easy-impossible  (boring:  3.7;  interesting:  3.0).  One  possible  explanation  is  that  when  learners  feel 
competent,  complexity  and  difficulty  (challenge)  heighten  interest;  otherwise  they  contribute  to  boredom. 

Research  Question  #3  (What  are  the  dimensions  of  learning  interest  and  boredom  as  they  relate  to  emotion?)  was 
explored  through  administration  of  a second  scale  to  the  same  162  subjects.  This  scale,  based  on  similar  scales 
developed  by  Russell  (1978)  (with  reliabilities  ranging  from  .70  to  .95)  consisted  of  a set  of  48  items  each 
containing  two  non-bipolar  terms  (e.g.  dull-troubled,  friendly-scornful).  Items  contained  terms  randomly  selected 
from  a list  of  69  terms,  which  meant  that  some  terms  appeared  more  than  once  in  this  scale. 

Subjects  were  asked  to  describe  their  feelings  by  placing  a check-mark  somewhere  along  a nine-point  line  between 
the  two  terms,  as  in  the  example  below. 

Cruel : : : : : : : : Hapny 

Each  subject  was  asked  to  rates  both  their  interesting  situation  and  their  boring  situation  using  this  set  of  terms. 

Before  these  data  were  analyzed,  a set  of  assumptions  were  made  about  the  process  subjects  used  to  complete  this 
task.  It  was  assumed  that  the  subject  would  first  choose  one  of  the  two  terms  as  more  descriptive  than  the  other  of 
each  of  their  situations.  Then,  based  upon  the  descriptive  strength  of  the  term  selected,  the  subject  would  select  a 
position  along  the  scale.  For  example,  in  the  item  cruel-happy , the  subject  might  believe  that  he  felt  more  happy 
than  cruel , decide  to  what  degree  of  happy  he  felt,  and  place  a check-mark  in  one  of  the  four  spaces  closest  to  the 
term  happy  as  in  the  example  below. 

Cruel : : : : : : : __x_  : Happy 
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Furthermore  since,  generally  speaking,  the  terms  were  not  antonyms,  we  could  not  assume  the  score  reflected  the 
degree  on  two  terms;  i.e.,  if  the  subject  had  placed  a check-mark  on  the  space  closest  to  happy  it  might  be  interpreted 
that  the  subject  was  at  the  highest  degree  of  happy  but  could  not  be  interpreted  that  he  also  felt  at  the  lowest  degree 
of  cruel.  It  was  assumed  that  once  the  subject  chose  happy , he  would  only  concentrate  on  how  to  quantify  his  happy 
degree  and  didn’t  care  about  his  cruel  degree.  The  check-mark  located  on  one  side  of  the  scale  or  the  other  reflects  1 ) 
the  strength  of  this  term  alone  as  a suitable  descriptor  and  2)  the  failure  of  the  other  term  to  be  chosen,  with  no 
designation  of  failure  strength. 

Based  on  the  above  rationale,  all  48  items  were  broken  into  two  one-term  measures  and  scored  according  to  the  four 
closest  spaces  to  that  term.  The  middle  space  was  coded  as  0 for  both  terms.  Two  examples  appear  below  (F  means 
failure  to  be  chosen) 


Cruel  _1_  : _2_  : _3_:  _4_  : _0_  : _F_  : _F_  : _F_  : _F_  Happy 
Cruel  _F„  : _F_  : _F_J  _F_  : _0_  : _4_  : _3_  : _2_  : _1_  Happy 


The  ten  highest-  and  lowest-scoring  terms  were  determined  for  both  the  boring  and  the  interesting  situations. 
Repeated  terms  were  removed  from  each  list  leaving  six  highest-scoring  terms  and  ten  lowest-scoring  terms  for  the 
interesting  situations  and  nine  highest-scoring  terms  and  eight  lowest-scoring  terms  for  boring  situations.  (The  terms 
and  their  scores  appear  in  Table  6).  Four  highest-scoring  terms  for  interesting  situations  were  among  the  lowest- 
scoring  terms  for  boring  ones  while  two  highest-scoring  terms  for  boring  situations  were  among  the  lowest-scoring 
terms  for  interesting  situations. 


Interesting 

(highest-scoring) 

Boring  (highest-scoring) 

Term 

Score 

Term 

Score 

interested 

587 

uninterested 

479 

wide-awake 

532 

unmotivated 

417 

happy* 

496 

frustrated 

406 

alert 

481 

angry* 

355 

friendly* 

476 

controlled 

286 

excited** 

433 

dull** 

279 

distressed* 

264 

adverse 

261 

nasty* 

255 

Interesting 

(lowest-scoring) 

Boring  (lowest-scoring) 

Term 

Score 

Term 

* 

Score 

enraged 

-149 

hopeful 

-97 

vicious 

-149 

friendly* 

-97 

spiteful 

-153 

excij[ed* 

-98 

scornful 

-154 

affectionate*  • 

-109 

cruel*** 

' -156 

wide-awake 

-113 

adverse 

-156 

interested 

-126 

hateful 

-1^8 

sad* 

-129 

tyrannical 

-159 

calm 

-138 

nasty* 

-160 

angry* 

-162 

Table  6.  Highest-  and  Lowest-scoring  Terms  for  Interesting  ahd  Boring  Situations 

[*  appears  twice:  **  appears  three  times;  ***  appears  four  times} 
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The  results  of  this  analysis  indicate  that  subjects  tended  to  use  more  extreme  ranks  to  describe  interesting  (433-587) 
than  boring  (255-479)  situations.  Two  possible  explanations  are:  (1)  the  experimental  situation  was  above  average; 
i.e,  the  distance  between  subjects*  interest  feeling  and  their  expectation  of  interest  was  shorter  than  the  distance 
between  their  boredom  feeling  and  their  expectation  of  boredom  and  (2)  subjects  were  more  reluctant  to  rate  negative 
terms  highly  than  to  rate  positive  terms  highly. 

Analyses  (i.e.,  Pearson  r correlation,  Pearson  r square)  were  conducted  on  each  word  term  pair  in  order  to  yield  a 
coefficient  of  nondetermination.  A coefficient  of  nondetermination  reflects  that  proportion  of  the  variance  which 
cannot  be  predicted  by  or  determined  by  the  other  variable.  Those  word-pairs  with  a coefficient  of  nondetermination 
value  equal  to  or  greater  than  0.90  were  selected,  yielding  34  orthogonal  measure  pairs.  Similar  terms  were  grouped 
together  linguistically  producing  the  boredom-interest  model  shown  in  Figure  1.  This  model  is  consistent  with 
previous  research  on  emotion.  The  terms  along  the  horizontal  axis  reflect  the  Arousal  dimension,  while  those  along 
the  vertical  axis  reflect  the  Pleasure  dimension.  The  third  axis  reflects  both  factors  (competence  and  self- 
determination)  along  the  Potency  dimension. 


Friendly  (18, 45) 

Happy  (2) 

Controlled  (4) 


Interested  (69) 
Wide-awake  (34, 53) 
Excited  (42) 


Controlling  (3) 


Angry  (9) 
Adverse  (17) 
Nasty  (6) 
Frustrated  (84) 
Distressed  (23) 


Figure  1.  Boredom-Interest  Model  for  Instructional  Situations 


Conclusions 

This  study  explored  the  various  emotions  related  to  boring  and  interesting  learning  experiences,  the  impact  of 
specific  sources  (i.e.  instructor,  learner,  materials,  environment)  on  resulting  boredom  or  interest,  and  the  influence 
of  a range  of  instructional  characteristics  (e.g.  predictability,  complexity)  on  student  perceptions  of  boring  and 
interesting  instruction. 

One  interpretation  of  the  results  indicates  that  feelings  of  pleasure  and  arousal  are  linked  to  generating  and  sustaining 
current  learning  interest  while  competence  and  self-determination  are  more  closely  related  to  fostering  a continuing 
motivation  to  learn  (Dodge,  1990). 


There  are  potential  applications  of  this  type  of  research  to  the  development  of  more  motivating  instructional 
materials  and  interactive  computer-based  learning  systems.  For  example*  results  of  this  study  may  also  be  used  as 
motivational  guidelines  for  the  design  of  effective  instruction.  Specifically: 

• Colorful  instruction  that  incorporates  a variety  of  attention-gaining  and  maintaining  strategies  appears  to  be 
the  most  effective  for  generating  interest  and  preventing  boredom. 

• Instruction  that  incorporates  surprise,  novelty  and  variety  (the  major  element  of  Keller's  Attention 
Component)  may  help  reduce  the  predictability  that  appears  to  promote  learning  boredom. 

• Instructional  materials  (e.g.  videos)  that  do  not  capture  students’  attention  and  are  not  directly  relevant  to  the 
content  and  goals  of  the  instruction  may  promote  boredom. 

• Instructors  are  perceived  by  learners  as  having  the  prime  responsibility  for  learner  interest  or  boredom. 
Therefore,  instruction  must  be  designed  to  include  motivational  strategies  that  enhance  learner  interest  and 
reduce  learner  boredom.  At  the  same  time,  instructors  need  to  incorporate  ways  to  help  learners  take  more 
responsibility  for  their  own  motivation,  as  currently  learners  do  not  perceive  themselves  as  potentially 
influencing  their  own  boredom  levels. 

One  of  the  underlying  goals  of  instruction  is  to  motivate  students  to  learn.  Designers  of  instruction  strive  to 
motivate  students  to:  (1)  want  to  learn,  (2)  enjoy  learning  while  it  is  occurring,  and  (3)  continue  learning  after  the 
instruction  has  formally  ended.  Research  that  illuminates  what  instructional  factors  promote  interest  and  reduce 
boredom  will  move  us  closer  toward  our  goal. 
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Although  it  is  not  unusual  for  R.M.  Gagnes  work  to  be  considered  in  a volume  addressing  learning  theories,  his 
contributions  can  most  appropriately  be  considered  as  an  “instructional  theory.”  An  instructional  theory  is  an  integrated 
set  of  principles,  based  upon  learning  theory,  other  relevant  theories,  and  sound  replicable  research,  that  permits  one  to 
predict  the  effects  of  specific  instructional  conditions  on  a learner's  cognitive  processing  and  the  resulting  learned 
capabilities.  Gagnd  (1985)  described  the  nature  of  an  instructional  theory  as  an  “attempt  to  relate  the  external  events  of 
instruction  to  the  outcomes  of  learning  by  showing  how  these  events  lead  to  appropriate  support  or  enhancement  of 
internal  learning  processes.  . . The  province  of  an  instructional  theory  is  to  propose  a rationally  based  relationship 
between  instructionAL  events,  their  effects  on  learning  processes,  and  the  learning  outcomes  that  are  produced  as  a result 
of  these  processes."  (p.  244)  How  does  instructional  theory  relate  to  learning  theories,  instructional  psychology,  and 
instructional  design  models.  In  contrast  to  instructional  theories  that  tend  to  be  predictive  and  prescriptive,  learning 
theories  are  typically  descriptive  and  explanatory.  According  to  Driscoll  (1994)  a learning  theory  "a  set  of  constructs 
linking  observed  changes  in  performance  with  what  is  thought  to  bring  about  those  changes."  (p.  9)  Instructional 
psychology  is  the  study  of  the  facilitation  of  human  learning  through  instruction  and  can  result  in  instructional  design 
theories  and  models.  Instructional  design  models  employ  instructional  theories  to  prescribe  types  and  levels  of 
instructional  support  to  optimize  the  achievement  of  identified  learning  goals. 

Snow  and  Swanson  (1992)  suggested  that  the  components  of  an  instructional  theory  are  "a)  description  of  desired 
end  states  or  goals  of  instruction  in  a domain;  b)  description  of  goal-relevant  initial  states  of  learners  prior  to  instruction; 
c)  explication  of  the  transition  processes  from  initial  to  desired  states;  d)  specification  of  instructional  conditions  that 
promote  this  transition;  e)  assessment  of  performance  and  instructional  effects."  (p.  584)  If  we  compare  Gagnes 
instructional  theory  to  these  components  we  find  that  Gagnd’s  theory  does  have  these  components.  For  example,  Gagne 
describes  potential  end  goal  states  in  his  categorization  of  learning  capabilities.  These  goal  states  are  generic  in  that 
they  can  apply  across  a variety  of  content  areas.  For  each  of  the  goal  types  Gagnd  described  goal-relevant  initial 
states,  prerequisite  relationship  of  intellectual  skills  and  relationships  of  other  types  of  learning.  Gagn6  interpreted 
information  processing  theory  to  explicate  the  transition  processes  from  initial  to  goal  states  for  each  type  of 
learning.  Gagne's  greatest  impact  on  instructional  theory  may  be  his  thoroughness  in  specifying  instructional 
conditions  to  support  this  transition  process.  He  described  these  instructional  conditions  both  as  generalized  events  of 
instruction  and  as  specific  conditions  of  learning  for  each  type  of  learning  capability  (Conditions  of  Learning,  1965, 

1970,  1977,  1985)  Finally,  Gagnd  and  his  colleagues  extended  his  thorough  explication  of  learning  outcomes  into 
recommendations  for  assessment  within  each  category  (Gagn6  & Beard,  1978;  Gagnd,  Briggs,  & Wager,  1992). 

Although  Gagnd  was  the  first  theorist  to  bring  this  elements  together  into  an  instructional  theory,  as  with  all 
learning/instructional  theorists,  his  work  was  strongly  influenced  by  the  theorists  who  preceded  him.  To  gain  an  adequate 
perspective  of  Gagnd’s  influence  on  instructional  theory,  we  must  first  survey  the  status  of  instructional  theory  prior  to 
Gagnd's  influence. 

Status  of  Instructional  Theory  Before  Gagne 

The  need  for  instructional  theory  has  long  been  recognized,  and  as  early  as  1899,  William  James  pointed  out  that,  as 
important  as  psychology  is  to  education,  it  is  not  something  from  which  the  nature  of  the  instruction  may  be  directly 
induced:  “You  make  a great,  a very  great  mistake,  if  you  think  that  psychology,  being  a science  of  the  mind's  laws,  is 
something  from  which  you  can  deduce  definite  programmes  and  schemes  and  methods  of  instruction  for  immediate 
schoolroom  use”  (James,  1899/1958.  p.  23).  Instructional  theory  remained  an  elusive  topic  before  Gagnd’s 
contributions  of  the  early  I960’s.  Two  primary  avenues  of  thought  regarding  instructional  theory  in  the  decade  or  so 
prior  to  Gagnd’s  major  contributions  were  a focus  on:  a)  sequence  and  content  concerns  from  within  a curriculum  theory 
frame  of  reference,  and  b)  application  of  learning  theory,  particularly  applications  within  a programmed  instruction  frame 
of  reference. 

Curriculum  Theory,  Much  work  that  characterizes  the  status  of  instructional  theory  before  Gagne  is  seen  in 
the  work  of  curriculum  theorists.  People  such  as  Bruner  and  Tyler  are  among  those  whose  work  concentrated  on  matters 
of  sequence  and  the  content  of  learning.  Bruner’s  (1960)  concept  of  the  spiral  curriculum  and  Tyler’s  conception  of  the 
“rationale”  for  a course  as  its  design  beginning  point  (1950)  are  examples  of  curriculum  thinkers’  contributions  to 
instructional  thinking. 
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One  persistent  pattern  in  curriculum  thinkers'  approaches  to  instruction  is  to  place  primary  emphasis  upon! 
teaching.  Hosford  (1973),  for  example,  defined  curriculum,  instruction,  and  teaching  much  as  an  instructional  s> 
specialist  or  instructional  technologist  might,  but  he  insisted  on  placing  teaching  at  the  center  of  his  conception^ 
instruction  and,  in  subsequent  treatment  of  a theory  of  instruction,  made  the  continued  assumption  that  teaching  ( 
teachers  would  be  the  primary  (or  only)  means  of  implementation.  Such  a focus  on  teaching,  it  appears,  prevenj 
curriculum  theorists  from  thinking  vigorously  and  directly  about  instruction  itself.  Nonetheless,  for  better  or  wyse,  the 
more  philosophical  contributions  from  curriculum  thinkers  formed  a substantial  proportion  of  instructional  theo^ 


Bruner's  widely  recognized  work  in  instructional  theory  (1968)  proposed  four  criteria  that  an  instructional  Sieory 
should  meet.  An  adequate  theory  of  instruction,  according  to  Bruner,  would  provide  the  basis  for  specification  of  a) 
experiences  which  will  induce  motivation  to  learn,  b)  optimal  structures  of  knowledge  for  learning,  c)  optimal  sciences 
of  encounter;  and  d)  the  nature  and  pacing  of  rewards  and  punishments.  In  many  regards  Bruner's  widely  heraldeAvork  i 
seems  now  naive  and  quaint.  His  concentration  on  structures  of  knowledge  within  disciplines  at  the  expense  of  katment 
of  form  of  encounter  and  his  treatment  of  intrinsic  motivation  within  a category  labeled  “rewards  and  punishment^’  are  « 
reflections  of  a teaching-centered  view  of  instruction. 

In  a less  widely  recognized  but  at  least  equally  valuable  contemporary  work  to  Bruner's,  Gordon  (1968)  presented  a I 
relatively  mature  view  of  instruction  and  instructional  theory.  Gordon  defined  a theory  of  instruction  as  'a  set  of  * 

statements,  based  on  replicable  research,  which  would  permit  one  to  predict  how  particular  changes  in  the  educational  | 

environment  (classroom  setting)  would  affect  pupil  learning."  (p.  3)  Gordon  differentiated  the  terms  “instruction"  and  | 
“teaching"  by  noting  that  teaching  “refers  primarily  to  the  human  interaction  between  teacher  and  pupil"  (p.  3)  and 
instruction  as  the  more  encompassing  term,  referring  to  “the  activity  which  takes  place  during  instruction  and  wflfcin  the 
classroom  setting.  The  term  includes  both  material  and  human  variables."  (p.  3).  The  distinctions  that  Gordon  made 
between  instruction  and  teaching  are  useful  ones,  as  the  study  of  instruction  and  the  study  of  teaching  are  reflected  I 
separate  bodies  of  literature  as  well  as  distinct  traditions  of  interest  and  inquiry.  However,  reflecting  the  curriculum  and  " 
teaching  methods  orientation,  Gordon  restricted  his  conception  of  instruction  to  classroom  activities,  a restriction  that 
might  be  viewed  as  limiting  by  current  instructional  theorists. 


Before  Gagnd's  work  had  become  widely  recognized  (and,  of  course,  remaining  conventional  in  many  Education  ^ 
specialties  today)  many  authorities  believed  that  the  most  important  instructional  considerations  lay  within  the  structures 
of  subject  matter  disciplines  and  the  interface  of  those  structures  with  broad  developmental  characteristics  of  learners.  The 
structures  of  knowledge  within  the  various  disciplines-science,  mathematics,  history,  and  so  on-was  (and  is)  seen  to  ^ 
vary  radically  from  discipline  to  discipline  in  conceptual,  syntactical,  and  substantive  ways  (Ford  & Pugno,  1964*Phi  * 
Delta  Kappa,  1964).  Gagnd  brought  a sufficiency  of  scholarship  to  questions  of  learning  from  instruction  that  arise  f 
from  the  psychological  requirements  of  learning  tasks,  as  opposed  to  questions  which  arise  from  parent  disciplines  from 
which  subject  matter  comes,  to  yield  prescriptive  principles  which— though  not  universally  adopted  in  educational  theory^ 
have  had  a substantial  impact  upon  theory  and  research  that  have  examined  educational  practice.  ^ 

Applied  Learning  Theory,  Although  typically  involving  itself  with  animal  conditioning  experiments,  the  ♦ 
mainstreams  of  learning  psychology  in  the  first  half  of  the  twentieth  century,  exemplified  hv  Guthrie,  Skinner,  and  Hull,  , 
were  deeply  concerned  with  humr.l  learning.  Guthrie's  association-centered  theory  (Guthrie,  1935,  1942)  gave  rise  to  ' 
Sheffield's  (1961)  work  in  learning  complex  sequential  tasks  and  Lumsdaine's  (1961)  training  research  on  effects  of 
cueing.  Skinner's  operant  conditioning  saw  application  by  Skinner  (1954)  and  Holland  (1960).  Hull's  detailed, 
systematic,  and  quantified  approach  to  learning  based  on  drive-reduction  led  to  instructionally-relevant  research  on 
feedback  by  Miller  and  Dollard  (1941).  4 


Clearly,  learning  theory  was  in  disarray  when  Hilgard  wrote  the  concluding  chapter  to  Theories  of  Learning 
(Hilgard,  1948):  “We  need  a more  careful  delineation  of  the  kinds  of  learning  which  take  place....  This  search  for  the 
appropriate  concepts  is  not  merely  an  exercise  in  definition  or  classification.  It  requires  a high  order  of  theory  ^ 

construction,  based  on  open-minded  acceptance  of  demonstrable  relationships."  (p.  326-327)  Almost  30  years  later,  the 
concluding  chapter  in  the  fourth  edition  of  that  work  (Hilgard  & Bower,  1978)  was  entitled  “Instructional  Theory."  The 
first  reference  cited  in  that  chapter  is  the  first  review  of  instructional  psychology  in  Annual  Reviews  by  Gagnd  and  1 
Rohwer  (1969).  Hilgard  and  Bower  described  Gagnd’s  work  to  that  time  as  one  of  three  models  that  provide  “indications 
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of  what  is  to  come.”  (1978,  p.  614).  In  addition  to  Gagn6’s  “hierarchical  theory,”  Bruner’s  “cognitive-developmental 
theory”  and  Atkinson’s  “decision-theoretic  analysis  for  optimizing  learning”  are  described.  Of  these,  although  important 
subsequent  work  was  done  by  all  three,  Gagn6  appears  to  have  gone  the  furthest  toward  development  of  a full 
instructional  theory. 

A great  deal  of  interest  in  the  1950’s  was  generated  by  the  innovation  called  “programmed  instruction.”  Embodied 
in  both  teaching-machine  and  text-based  forms,  programmed  instruction  carried  with  it  ideas  of  far  greater  importance  than 
the  competing  specific  forms  and  rules  dictating  format  that  were  matters  of  heated  debate  at  the  time.  With  programmed 
instruction,  an  agency  other  than  a person  was  seen  as  an  instrument  of  instruction.  Previously,  all  non-human  tools 
including  books,  television,  and  the  various  forms  of  audiovisual  media,  were  conceptualized  as  aids  or  resources  for  a 
teacher’s  use.  Even  the  powerful  medium  of  motion  pictures  (and  later  television)  was  viewed  as  something  that  had 
“classroom”  uses,  which  required  a teacher’s  introduction  and  follow-up  for  meaningful  learning  to  anticipated. 

This  radical  change  in  view  of  the  potential  of  instructional  media  brought  with  it  a radical  change  in  what  might  be 
studied  as  “instruction”  and  how  research  on  it  might  be  conducted.  In  his  landmark  review,  “Instruments  and  Media  of 
Instruction,”  Lumsdaine  (1963)  pointed  out  the  significance  of  the  ideas  behind  programmed  instruction  for  thinking 
about  research  on  instruction:  “...the  control  of  learner  behavior  and  feedback  which  is  provided  by  the  continuous  record 
of  student  response  from  auto-instructional  programs  may  afford  the  most  promising  vehicle  yet  developed  for  the 
analytic  experimental  student  of  variables  affecting  human  learning  (as  well  as  for  the  incorporation  of  research  findings 
in  improved  instruments).”  (Lumsdaine,  1963,  p.  608)  Research  on  programmed  instruction  brought  with  it,  perhaps 
unknowingly  but  certainly  inevitably,  focused  concern  on  matters  of  form  of  encounter  with  material  to  be  learned 
outside  the  frame  of  reference  of  “teaching.”  As  teaching  itself  is  not  a reproducible  event,  the  study  of  teaching  has 
focused  on  matters  that  can  be  of  ultimate  utility  in  understanding  the  role  of  teachers,  understanding  the  teaching  act, 
and  understanding  interactions  among  teachers,  learners,  and  activities.  When  studying  instruction  using  agencies 
providing  sequenced  and  reproducible  events,  controlled  investigations  of  form  of  encounter  became  practicable.  An 
enormous  corpus  of  research  was  developed  during  the  1950’s  and  I960’s  under  the  programmed  instruction  umbrella  on 
areas  such  as  practice,  feedback,  sequence,  and  criterion-referenced  assessment. 

One  example  of  how  work  in  programmed  instruction  contributed  to  instructional  theory  can  be  seen  in  “validation” 
procedures  for  programmed  instruction.  The  first  work  in  what  would  now  be  labeled  “formative  evaluation”  was 
developed  under  the  notion  of  “how  should  programmed  instruction  be  validated?”  Procedures  for  the  development  and 
establishment  of  known  quality  in  programmed  instruction  materials  have  evolved  over  the  years  to  include  instruction  in 
any  form  and  are  the  basis  of  current  formative  evaluation  principles  and  procedures.  Other  examples  can  be  seen  in 
studies  on  such  areas  as  instructional  feedback,  instructional  event  sequencing,  pacing,  optimal  prompting  of  practice,  and 
forms  of  practice  and  response. 

Other  cornerstones  of  contemporary  instructional  design  practice,  such  as  use  of  performance  objectives,  have  their 
roots  in  programmed  instruction.  However,  the  generalization  which  transpired  between  thinking  of  “how  to  best 
implement  a particular  programmed  instruction  format”  on  the  one  hand  and  “how  to  optimally  conduct  instruction”  on 
the  other,  is  not  a trivial  one.  Gradually  attention  began  to  shift  from  the  procedural  details  to  variables,  questions,  and 
models  of  instruction  that  would  underlie  the  techniques.  No  one  contributed  more  to  this  shift  in  thinking— much  of  his 
own  work  in  the  1950’s  and  early  1960’s  can  be  seen  as  an  embodiment  of  it-than  Robert  Gagn6. 

Precursors  to  Instructional  Theory 

Of  the  schools  of  thought  that  underlie  instructional  theory,  Gagn6  clearly  comes  from  the  “applied  science” 
perspective.  As  a psychologist,  he  studied  learning  in  demanding,  realistic  settings,  and  was,  in  fact,  somewhat 
impatient  with  colleagues  whose  purity  of  purpose  prevented  their  doing  the  more  messy  and  often  less  clear  applied 
research  that  instructional  theory-building  requires.  In  a review  of  factors  that  contribute  to  learning  efficiency  for  a 
volume  on  programmed  instruction  sponsored  by  the  Air  Force  Office  of  Scientific  Research,  Gagn6  and  Bolles  noted 
that  “the  learning  tasks  that  have  been  most  intensively  studied  by  psychologists  have  been  of  an  artificial  “laboratory” 
variety;  relatively  little  is  known  about  learning  in  real  life  situations.”  (Gagn6  & Bolles,  1959,  13-14) 


730 


Training  research  in  the  1950’s  put  Gagn6  in  touch  with  a wide  variety  of  instructional  problems,  representing  a 
wide  variety  of  learning  tasks.  Illustrative  studies  in  the  literature  are  Gagn£  (1954)  “An  Analysis  of  Two  Problem 
Solving  Activities”  involving  troubleshooting  and  interpretation  of  aerial  photographs,  and  Gagn£,  Baker,  & Wylie 
(1951)  “Effects  of  an  Interfering  Task  on  the  Learning  of  a Complex  Motor  Skill”  involving  manipulations  of  controls 
similar  to  aircraft  controls.  In  a review  of  problem-solving  and  thinking,  Gagn6  pointed  out  the  relevance  of  trouble 
shooting  studies  to  issues  in  concept  formation  (Gagn6,  1959).  Wide  and  vigorous  participation  in  research  on  learning 
and  instruction  in  the  military  environment,  along  with  his  thorough  and  rigorous  background  as  a learning  psychologist, 
may  have  created  the  dissonance  that  motivated  Gagn6  to  develop  the  concepts  of  conditions  of  learning,  learning 
hierarchies,  events  of  instruction,  and  types  of  learning.  In  the  following  pages,  we  will  discuss  each  of  these  three 
contributions  to  instructional  theory. 

Instructional  Theory  Contributions 

Gagn6  developed  four  major  propositions  that  constitute  his  instructional  theory: 

a)  Learning  goals  can  be  categorized  as  to  learning  outcome  or  knowledge  type; 

b)  Acquisition  of  different  outcome  categories  requires  different  internal  processes; 

c)  Learning  outcomes  can  be  represented  in  a predictable  prerequisite  relationship; 

d)  Acquisition  of  different  outcome  categories  requires  identifiably  different  instructional  processes. 

Development  of  Types  of  Learning.  Gagn6  was,  of  course,  not  the  first  theorist  to  suggest  that  all 
learning  is  not  alike,  that  learning  might  be  analyzed  into  different  types  of  learning.  Indeed,  as  early  as  1933  Can- 
suggested  classes  of  experimental  learning  tasks  and  warned  that  principles  that  had  been  derived  about  one  set  of  tasks 
could  not  necessarily  be  generalized  to  other  classes  (Melton,  1964).  In  the  1940’s  scientists  such  as  Melton  (1941)  and 
Tolman  (1949)  continued  the  efforts  to  categorize  learning  types.  During  an  informal  meeting  of  college  examiners  at 
the  1948  American  Psychological  Association  conference.  Bloom  and  his  colleagues  discussed  the  need  for  a set  of 
common  descriptors  of  learning  to  facilitate  communication  among  them.  This  effort  resulted  in  "Bloom’s  Taxonomy" 
of  cognitive  educational  objectives  (Bloom,  Englehart,  Furst,  Hill,  & Krathwohl,  1956)  and  Krathwohl’s  taxonomy  of 
affective  educational  objectives  (Krathwohl,  Bloom,  & Masia,  1964).  Despite  the  original  intention  of  these  taxonomies 
to  standardize  terminology,  they  readily  assumed  the  stature  of  psychologically-based  correlates. 

In  1962,  Melton  organized  a "Symposium  on  the  Psychology  of  Human  Learning"  that  was  held  at  the  University 
of  Michigan  at  Ann  Arbor.  The  focus  of  the  symposium  was  to  discuss  "the  interrelationship  of  different  categories  of 
human  learning."  (p.  vii)  Melton  later  edited  a book  Categories  of  Human  Learning  (1964)  that  compiled  many  of  the 
papers  from  this  symposium.  Among  these  was  Robert  Gagne’s  chapter  "Problem  Solving."  In  this  chapter  Gagn6 
presented  a table  entitled  “A  Suggested  Ordering  of  the  Types  of  Human  Learning”  in  which  he  proposed  the  following 
six  types  of  learning:  Response  learning,  chaining,  verbal  learning  (paired-associates),  concept  learning,  principle 
learning,  problem  solving  (Gagn6,  1964,  p.  312).  Gagn£  did  not  cite  a previous  publication  related  to  these  concepts,  so 
this  chapter  may  be  the  First  appearance  of  his  types  of  learning  outcomes  categories.  In  this  chapter  he  began  to 
differentiate  between  verbal  learning  and  "nonreproductive"  types  of  learning,  such  as  concept  learning  and  problem 
solving.  This  differentiation  eventually  led  to  his  separate  domains  of  learning  of  "verbal  learning"  and  "intellectual 
skills." 


Gagne  presented  the  First  complete  statement  of  the  types  of  learning  conception  in  his  First  edition  of  The 
Conditions  of  Learning  (Gagn6;  1965).  He  began  by  reviewing  learning  theory  and  research,  such  as  James,  Dewey, 
Watson,  Thorndike,  Tolman,  Ebbinghaus,  Pavlov,  and  Kohler,  introducing  the  idea  of  types  of  learning,  with  the  notion 
of  “learning  prototypes,” 

Throughout  the  period  of  scientiFic  investigation  of  learning  there  has  been  frequent  recourse  to  certain  typical 
experimental  situations  to  serve  as  prototypes  for  learning.  . . . These  learning  prototypes  all  have  a similar 
history  in  this  respect:  each  of  them  started  to  be  a representative  of  a particular  variety  of  learning  situation. 
Thorndike  wanted  to  study  animal  association.  Pavlov  was  studying  reflexes.  Ebbinghaus  studied  the 
memorization  of  verbal  lists.  Kohler  was  studying  the  solving  of  problems  by  animals.  By  some  peculiar 
semantic  process,  these  examples  became  prototypes  of  learning,  and  thus  were  considered  to  represent  the 
domain  of  learning  as  a whole,  or  at  least  in  large  part.  (p.  18-19) 
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Gagn6  (1965)  presented  eight  types  of  learning  in  the  first  edition,  in  a rather  strict  hierarchical  relationship.  He 
described  all  types  but  the  first,  signal  learning  (classical  conditioning),  as  having  prerequisite  relationships  with  one 
another.  Gagn£  carefully  referenced  researchers  that  had  examined  these  eight  types  of  learning: 

1.  Signal  learning  (Pavlov,  1927) 

2.  Stimulus-Response  learning  (Thorndike,  1989;  Skinner,  1938;  Kimble,  1961) 

3.  Chaining  (Skinner,  1938;  Gilbert,  1962) 

4.  Verbal  Association  (Underwood,  1964) 

5.  Multiple  Discrimination  (Postman,  1961) 

6.  Concept  Learning  (Kendler,  1964) 

7.  Principle  Learning  (Gagn6,  1964) 

8.  Problem  Solving  (Katona,  1940;  Maier,  1930)  (pp.  58-59) 

This  list  remained  relatively  unchanged  in  the  second  edition  of  Conditions  of  Learning  (1970).  By  the  third 
edition  (Gagne,  1977),  Gagn<S  added  information  processing  theories  to  the  treatment  of  learning  prototypes,  recasting  the 
types  of  learning  to  some  degree  by  their  different  cognitive  demands  . In  addition,  an  increasing  influence  of  task 
characteristics,  rather  than  psychological  processes  guided  the  form  and  content  of  the  types  of  learning.  In  its  latest  form 
as  of  this  writing  (Fourth  Edition,  Gagn6,  1985),  he  identified  distinctly  different  categories  within  the  domain  of 
intellectual  skills:  discriminations,  concepts,  rules,  and  higher-order  rules  (domain -specific  problem-solving).  He 
proposed  that  these  knowledge  types  are  in  a prerequisite,  vertical  transfer  relationship,  with  discriminations  prerequisite 
to  concepts,  concepts,  prerequisite  to  rules,  and  rules  prerequisite  to  problem  solving.  The  types  of  learning  in  the  fourth 
edition  are: 

1.  Intellectual  Skills 

discriminations 

concepts 

rules 

problem  solving 

2.  Cognitive  Strategies 

3.  Verbal  Information 

4.  Motor  Skills 

5.  Attitudes 

(Gagne  later  pointed  out  that  the  verbal  information  category  could  also  be  termed  “Declarative  knowledge"  and  the 
intellectual  skills  category  could  be  termed  "procedural  knowledge.") 

More  recently,  Gagne  and  Merrill  (1990)  identified  another  category,  which  they  termed  "enterprise."  They 
described  this  category  as  being  substantively  different  from  the  other  learning  outcomes  that  Gagne  or  Merrill  had 
previously  identified,  requiring  the  integration  of  the  other  more  simple  learning  outcomes,  such  as  rules,  concepts,  and 
declarative  knowledge. 

There  are  two  ways  in  which  one  might  view  these  outcomes:  as  descriptions  of  tasks  (with  external,  “task-related’' 
differences)  or  are  they  descriptions  of  learned  abilities  (with  differences  arising  out  of  distinctive  processing  or  memory 
structures)?  Gagn6  has  tended  to  define  these  outcomes  as  the  latter,  describing  them  as  "learned  dispositions," 
"capabilities,"  or  "long  term  memory  states,"  (1985,  p.  245).  He  described  verbal  information  and  intellectual  skills  as 
having  distinctly  different  memory  storage  systems,  consistent  with  those  of  other  theorists,  such  as  Anderson  (1990). 
An  empirical  basis  for  the  "verbal  information"  knowledge  to  be  stored  as  propositional  networks  is  provided  by  Gagn6 
and  White  (1978).  Rule-using  is  described  by  Gagn6  and  White  as  stored  in  hierarchical  skill  structures  which  they 
referred  to  as  "intellectual  skills."  Verbal  information  has  been  described  more  recently  by  Gagn6  (1985)  as  being  stored 
as  propositional  networks  or  schemata.  He  described  rules,  including  concepts  (defining  rules),  as  being  stored  as  "if., 
then"  productions.  The  storage  of  problem  solving  capabilities  themselves  was  not  addressed,  although  interconnections 


of  schemata  and  productions  were  implied.  The  storage  mechanisms  of  attitudes,  motor  skills,  or  cognitive  strategies  are 
also  not  explicitly  discussed. 

Gagne’s  categorization  of  learning  outcomes  has  not  been  without  it  critics.  Although  Gagn6  characterized  his 
categorization  system  as  more  internally,  than  externally  derived,  KyDonen  and  Schute  (1989)  viewed  Gagne's 
categorization  of  learning  types  as  a “rational  taxonomy,”  developed  via  proposing  “task  categories  in  terms  of 
characteristics  that  will  foster  or  inhibit  learned  performance.”  (p.  120)  They  suggested  that  the  limitation  of  such 
categories  is  that  their  basis  is  not  psychological  processes  and,  therefore,  such  processes  are  unsystematically 
considered. 

Development  of  the  Learning  Hierarchies  Concept.  Perhaps  as  significant  as  his  delineation  of 
categories  of  learning,  is  Gagne’s  conception  of  a learning  hierarchy.  Although  this  hierarchical  relationship  was  implied 
in  the  taxonomies  of  a number  of  theorists  (Cotterman,  1959;  Demaree,  1961 ; Lumsdaine,  1960;  Miller,  1962;  Parker  & 
Downs,  1961;  Stolurow,  1964;  Willis  and  Patterson,  1961),  it  was  Gagn6  who  brought  the  conception  of  ’’learning 
hierarchy”  clearly  into  focus  with  his  statements  regarding  the  nature  of  these  relationships  and  his  research  to  validate 
these  principles. 

Gagne’s  first  references  to  “learning  hierarchies”  appears  in  articles  published  in  1962,  a report  of  a study, 
“Factors  in  Acquiring  Knowledge  of  a Mathematical  Task”  (Gagn6,  Mayor,  Garstens,  and  Paradise,  1962  ) and  another 
study,  “The  Acquisition  of  Knowledge,”  (Gagn6,  1962)  which  involved  similar  learning  tasks.  These  reports  were 
preceded  by  a Gagne  and  Paradise’s  1961  study,  which  formed  a foundation  for  the  latter  studies.  In  1961 , Gagn6  and 
Paradise  found  support  for  the  proposition  that  transfer  of  learning  from  subordinate  sets  of  learning  tasks  could  account 
for  performance  in  a terminal  learning  task.  In  a subsequent  study,  Gagn6,  Mayor,  Garstens,  and  Paradise  ( 1962)  sought 
to  extend  and  confirm  the  validity  of  the  idea  of  the  “learning  hierarchy.”  In  this  study,  the  posttest  supplied  information 
about  achievement  of  not  only  the  terminal  task  (adding  integers)  but  also  the  12  prerequisite  learning  sets,  each  scored  as 
"pass"  or  "fail.”  Success  in  final  task  achievement  correlated  highly  with  the  number  of  subordinate  tasks  successfully 
achieved  for  both  of  the  two  terminal  learning  tasks  (.87  and  .88).  Patterns  of  transfer  among  the  subordinate  tasks  also 
conformed  to  theoretical  predictions  of  a learning  hierarchy. 

In  1973,  Gagn6  fully  described  learning  hierarchies  as  having  the  following  characteristics:  They  a)  describe 
“successively  achievable  intellectual  skills,  each  of  which  is  stated  as  a performance  class;  ” b)  do  not  include  “verbal 
information,  cognitive  strategies,  motivational  factors,  or  performance  sets;”  and  c)  describe  “only  those  prerequisite 
skills  that  must  be  recalled  at  the  moment  of  learning”  to  supply  the  necessary  “internal”  component  of  the  total  learning 
situation,  (p.  21-22) 

White  (1973)  reviewed  a number  of  studies  that  attempted  to  validate  learning  hierarchies  developed  according  to 
Gagne's  principles.  He  found  none  that  had  a perfect  match  with  predicted  prerequisite  relationships.  However,  he 
suggested  that  many  of  the  studies  were  seriously  flawed  by  imprecise  specification  of  prerequisite  tasks,  using  only  one 
item  per  prerequisite  task,  small  sample  sizes,  and  other  methodological  problems.  Research  has  continued  both  on 
methodologies  to  validate  hierarchies  and  techniques  for  specifying  hierarchies  (e.g.,  Airasian  & Bart,  1975;  Cotton, 
Gallagher,  & Marshall,  1977;  Griffins,  1983;  Kee  & White,  1979;  Wilson,  1989;  Winkles,  1986).  For  example, 
Winkles  (1986)  investigated  the  learning  of  trigonometry  skills  with  a learning  hierarchy  validation  study,  identifying 
both  lateral  and  vertical  transfer.  Two  experiments  with  eighth  and  ninth  -grade  students  involved  instructional  treatments 
described  as  “achievement  with  understanding”  and  “achievement  only.”  Results  reported  “achievement  with 
understanding  treatment  is  better  for  the  development  of  lateral  transfer  for  most  students,  and  of  vertical  transfer  for  the 
more  mathematically  able  students,  whereas  the  differences  between  the  treatment  groups  on  tests  of  achievement  and 
retention  of  taught  skills  are  not  significant.  A small  amount  of  additional  instruction  on  vertical  transfer  items  produces 
much  better  performance  under  both  treatments.”  (p.  275) 

Development  of  Events  of  Instruction.  In  1962,  in  addition  to  presenting  the  “learning  hierarchies” 
concept,  Gagn6  also  introduced  began  to  consider  features  that  should  be  included  in  the  instructional  situation,  such  as 
description  of  the  required  terminal  performance  and  provision  of  “guidance  of  thinking.”  Then,  in  the  first  edition  of 
The  Conditions  of  Learning  (1965)  Gagn6  included  a section  headed  “component  functions  of  the  instructional  situation” 


which,  except  for  its  label,  is  basically  identical  to  the  “events  of  instruction”  seen  in  later  editions  of  The  Condition?  of 
T /».nminp  The  eight  functions  were  a)  presenting  the  stimulus,  b)  directing  attention  and  other  learner  activities,  c) 
providing  a model  for  terminal  performance,  d)  furnishing  external  prompts,  e)  guiding  the  direction  of  thinking,  f) 
inducing  transfer  of  knowledge,  g)  assessing  learning  attainments,  and  h)  providing  feedback.  In  the  second  edition  of 
The  Conditions  of  I .earning  (Gagnd,  1970),  Gagnd  added  “The  Events  of  Instruction”  to  a new  chapter  entitled  ‘The 
Design  of  Instruction,”  completing  the  development  of  the  fundamental  concept  of  "the  events  of  instruction." 

Although  researchers  have  expended  much  effort  in  investigating  the  optimal  nature  of  individual  events  (e.g., 
feedback  research,  research  on  objectives),  the  validity  of  events  of  instruction  as  a whole  have  not  been  subjected  to 
much  research.  This  must  in  part  be  due  to  Gagne's  assertion  that  instruction  must  not  necessarily  include  all  events  on 
all  occasions,  as  learners  are  often  able  to  supply  the  processing  that  the  events  evoke  without  external  prompting.  The 
authors  did,  however,  find  one  study  that  examined  the  effectiveness  of  the  events  in  instruction  for  high  school  students 
on  the  use  of  quotation  marks  (Coats,  1985).  The  experimental  study  involved  three  treatments:  a)  all  nine  events;  b) 
only  four  events:  presenting  stimulus  materials,  providing  learning  guidance,  eliciting  performance  and  providing 
feedback,  and  c)  the  same  events  as  in  treatment  b with  more  elaborate  eliciting  performance  and  feedback  events.  The 
results  indicated  no  main  effects  for  treatments,  but  an  interaction  between  ability  and  treatment:  High  ability  learners 
performed  better  under  treatment  b;  low  ability  learners  under  treatment  c.  It  is  really  no  surprise  that  by  high  school  the 
students  did  not  need  much  introduction  to  quotation  marks,  that  are  represented  by  the  early  events  of  instruction.  Nor  is 
it  a surprise  that  the  low  ability  learners  performed  better  under  a condition  that  required  more  practice  and  feedback. 

Conditions  of  Learning.  Perhaps  more  than  the  explication  of  categories  of  learning  capabilities  and  the 
events  of  instruction,  Gagne's  major  contribution  to  instructional  theory  lies  in  his  suggestion  that  for  each  category  or 
subcategory  of  learning  capability  to  be  acquired,  certain  internal  conditions  must  be  met.  His  attention  to  the  conditions 
within  the  learner  has  been  long-lasting,  as  he  has  conjectured  about  necessary  conditions  within  the  learner  since  his  first 
edition  of  Conditions  of  learning  (19651.  He  further  suggested  from  this  first  edition  that  these  internal  conditions  vary 
somewhat  by  learning  capability.  Specifically,  he  has  proposed  in  more  recent  years  that  the  internal  events  that  may 
differ  most  across  learning  capabilities  are  "a)  substantive  type  of  relevant  prior  knowledge,  b)  manner  of  encoding  into 
long  term  storage,  c)  requirement  for  retrieval  and  transfer  to  new  situations."  (1984,  p.  514)  It  should  be  noted  that  in 
Gagn6's  detailing  of  the  internal  conditions  of  each  type  of  learning,  the  major  internal  condition  that  he  details  is 
prerequisite  knowledge.  Gagn6  used  an  information-processing  model  including  processes  of  attention,  selective 
perception,  semantic  encoding,  retrieval,  response  organization,  control  processes,  and  expectancies  to  contextualize  these 
cognitive  processes. 

He,  therefore,  in  his  1985  edition  of  Conditions  of  Learning  pointed  out  that  the  events  that  may  differ  most 
significantly  from  learning  category  to  learning  category  are  those  corresponding  to  the  above  three  internal  events: 

These  external  events  are  a)  stimulating  recall  of  prior  knowledge,  b)  providing  learning  guidance,  and  c)  enhancing 
retention  and  transfer. 

As  an  instructional  psychologist,  Gagn6  was  particularly  interested  in  the  external  conditions  that  might  occur  or 
could  be  provided  to  "activate  and  support"  the  internal  processing  necessary  for  learning  to  occur  (1985,  p.  276).  In 
fact,  Gagn6  defined  the  purpose  of  instructional  theory  as  "to  propose  a rationally  based  relationship  between  instructional 
events,  their  effects  on  learning  processes,  and  the  learning  outcomes  that  are  produced  as  a result  of  these  processes." 
(1985,  p.  244)  Therefore,  Gagne  derived  the  external  events  from  the  internal  events  of  information  processing. 

Gagn6  particularized  the  general  external  events,  the  “events  of  instruction,”  that  begin  to  be  described  in  his  work 
in  1962  to  specific  prescriptions  for  external  conditions  for  each  type  of  learning,  event  by  event,  for  each  of  the 
categories  of  learned  capability.  Much  of  these  external  conditions  is  logically  derived  from  the  intersection  of  the 
function  of  the  external  event  (those  cognitive  processes  that  it  supports)  and  the  nature  of  the  learning  capability.  He 
labeled  these  external  supports  for  each  type  of  learning  as  their  "(external)  conditions  of  learning." 

Gagn6  and  Glaser  (1987)  included  an  excellent  summary  of  hypothesized  differential  learning  conditions  for 
types  of  learning. 
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Table  7- 1 . Gagnd  and  Glasser’s  Learning  Categories  X Conditions  Summary 


Type  of  Capability 
Intellectual  skill 


Verbal  information 


Cognitive  strategy 
(problem  solving) 


Attitude 


Motor  Skill 


Effective  learning  conditions 
for  categories  of  learned  capabilities 

Learning  Conditions 

Retrieval  of  subordinate  (component)  skills 
Guidance  by  verbal  or  other  means 
Demonstration  of  applic.  by  student;  precise  feedback 
Spaced  reviews 

Retrieval  of  context  of  meaningful  information 
Performance  of  reconstructing  new  knowledge;  feedback 

Retrieval  of  relevant  rules  & concepts 
Successive  presentation  (usually  over  extended  time) 
of  novel  problem  situations 
Demonstration  of  solution  by  student 

Retrieval  of  information  and  intell.  skills 
relevant  to  targeted  personal  actions 
Establishment  or  recall  of  respect  for  human  mode! 
Reinforcement  for  personal  action  either  by  successful 
direct  experience  or  vicariously  by  observation  of 
espected  person 

Retrieval  of  component  motor  chains 
Establishment  or  recall  of  executive  sub-routines 
Practice  of  total  skill;  precise  feedback. 

(Gagne  & Glaser,  1987,  p.  64) 


Unfortunately,  there  has  not  been  systematic  research  investigating  the  validity  of  the  principles  for  external 
conditions  for  specific  types  of  instruction  as  suggested  by  Gagn6.  However,  there  are  some  lines  of  research  that  have 
suggested  that  his  underlying  premise  that  different  types  of  learning  are  facilitated  by  different  instructional  conditions. 
For  example,  a meta-analysis  by  Schimmel  (1983)  suggested  that  feedback  may  be  more  potent  for  intellectual  skills 
objectives  than  for  verbal  information  objectives.  In  addition,  Schimmel  found  that  confirmation  feedback  was  more 
useful  than  correct  answer  feedback  for  verbal  information  outcomes.  However,  he  did  not  find  this  superiority  of 
confirmation  feedback  for  intellectual  skills  objectives.  Research  on  the  value  of  providing  learners  with  objectives 
reveals  another  example  of  research  that  may  support  Gagnes  principles  of  conditions  of  learning.  Hartley  and  Davies 
(1976)  found  that  providing  objectives  benefited  students  when  the  learning  task  is  an  intellectual  skill,  but  was  not 
significantly  beneficial  to  promote  verbal  information  learning.  Although  these  findings  are  in  no  way  comprehensive, 
they  do  provide  some  validation  of  an  outcomes  by  conditions  theory  of  instruction. 

The  Relationship  of  Gagne's  Work  to  Learning  Theory 

Gagn6s  work  is  not  easily  related  to  a single  learning  theory  base.  His  early  work,  frequently  characterized  as 
behaviorist,  might  better  be  considered  more  broadly  as  association^.  Within  the  associationist  tradition,  his  work 
appears  to  fit  better  within  the  functionalist  group  than  either  the  behaviorist  or  connectionist.  However,  the  integrative 
nature  of  his  work,  particularly  as  reflected  in  the  first  edition  of  Conditions  of  Learning,  transcended  the  traditional 
categories  of  learning  theory.  An  examination  of  the  sources  for  the  eight  categories  of  learning  identified  in  this  first 
edition  reveals  origins  in  field  theory  (from  gestalt  psychology)  as  well  as  functionalism  and  behaviorism.  The  range  of 
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learning  types  that  he  wished  to  consider  were  not  adequately  examined  under  any  single  theory  at  that  time,  so  he  had  to 
examine  a number  of  learning  theories  in  order  to  develop  his  instructional  theory. 

Certainly,  by  the  first  edition  of  Conditions  of  Learning  (1965).  Gagne  was  beginning  to  conjecture  about  the 
internal  conditions  of  the  learner.  And,  by  the  second  edition,  he  used  an  information  processing  model  (although  it  is 
not  labeled  as  such)  to  describe  the  cognitive  processes  that  occur  during  a "learning  sequence."  (1970,  pp.  70-71)  In  the 
third  edition  of  Conditions  of  Learning  (1977).  Gagnd’s  instructional  theory  was  thoroughly  integrated  with  information 
processing  theory.  Although  he  employed  other  theories,  as  appropriate.  Gagne  has  continued  to  draw  substantially 
from  information  processing  theory,  one  of  the  family  of  cognitive  learning  theories,  as  his  basis  for  describing  the 
internal  processes  and  structures  of  learning  that  are  affected  by  the  external  conditions  and  events  of  instruction. 

One  frequently  finds  Gagnd’s  work  described  as  behaviorist,  associationist,  as  well  as  cognitivist.  Indeed,  it  is 
difficult  to  utilize  a single  learning  theory  with  which  to  characterize  his  work.  An  attribute  of  Gagnd's  theory  that 
makes  it  so  difficult  to  accurately  categorize  is  that  although  his  theor/  base  was  eclectic  in  source  it  was  unified  in 
result.  Hilgard  and  Bower  pointed  out  that  Gagnd’s  work  "is  not  strict  iy  an  eclectic  theory  (which  chooses  good 
principles  from  here  and  there  without  any  order  among  them),  but  is  the  beginning  of  a unified  theory.  " (1966,  p.  569). 
And  that  result,  an  instructional  theory,  even  in  its  earliest  versions  is  not  well-classified  as  behaviorist  or  even 
associationist  psychology.  Gagne's  theory  as  reflected  in  later  editions  of  Conditions  of  Learning  moved  even  further 
from  the  associationist  perspective  and  became  increasingly  based  on  cognitive  psychology,  with  an  information 
processing  emphasis.  Perhaps  the  source  of  the  difficulty  in  classifying  Gagne's  theory  arises  from  attempts  to  classify  it 
under  the  categorization  system  of  learning  theories,  when  it  is  in  effect  an  instructional  theory  that  proposes  facilitating 
instructional  conditions  for  a range  of  learning  types  from  declarative  knowledge  to  psychomotor  skills  to  attitudes.  No 
single  learning  theory  at  this  time  appears  to  adequately  explain  or  predict  all  of  these  types  of  learning.  Hence,  an 
eclectic  learning  theory  base  for  Gagne's  instructional  theory  is  entirely  appropriate. 

Influences  of  Gagne’s  Theory  on  Instructional  Design  Models 

Gagne's  theory  has  been  foundational  in  providing  the  basis  of  what  can  be  termed  "conditions-based"  models  of 
instructional  design.  Conditions-based  models  are  predicated  upon  the  propositions  that  (a)  learning  can  be  classified  into 
categories  that  require  similar  cognitive  processes  for  learning  ( "internal  conditions  of  learning")  and,  therefore,  (b) 
within  these  categories  of  learning  similar  instructional  supports  are  needed  to  facilitate  learning  ("external  conditions  of 
learning").  Conditions-based  models  of  design  which  are  derivative  of  Gagne’s  work  include  those  by  Merrill  (1983), 
Reigeluth  (1979),  Merrill,  Li,  and  Jones  (1990a  & b),  and  Smith  and  Ragan  (1993). 

In  the  early  1970’s,  M.D.  Merrill  developed  a conditions-based  model  for  instructional  design  called  “component 
display  theory.”  Merrill  noted  that  component  display  theory  evolved  from  his  interactions  with  students  studying  Gagne 
and  that  “CDT  is  founded  on  the  same  assumption  as  Gagnd’s  work-namely  that  there  are  different  categories  of 
outcomes  and  that  each  of  these  categories  requires  a different  procedure  for  assessing  achievement  and  a different 
procedure  for  promoting  the  capability  represented  by  the  category.”  (Merrill,  1983,  p.  284-285)  Early  work  describing 
most  of  the  elements  of  CDT  appears  in  Merrill  and  Boutwell  (1973).  CDT  classifies  objectives  in  a two-dimensional 
matrix  made  up  of  three  performance  levels  and  four  content  types.  This  12-category  system  differs  from  GagntS’s  in  that 
instead  of  having  a declarative  knowledge  category,  as  Gagne  does,  which  would  include  remembering  facts,  concept 
definitions,  rule  statements,  and  procedural  steps,  CDT  provides  separate  categories  for  each  of  these  types  of  declarative 
knowledge.  And,  instead  of  having  a single  category  for  cognitive  strategies,  as  Gagnd  does,  CDT  proposes  "find" 
operations  for  each  of  the  content  types:  Find  a fact,  find  a concept,  find  a rule,  and  find  a procedure.  CDT  also  includes 
a treatment  of  external  conditions  for  lear  ning  as  “presentation  forms,”  including  content  (generality  or  instance),  and 
approach  (expository  or  inquisitory)  as  primary  forms  and,  for  secondary  presentation  forms,  elaborations  such  as  context, 
prerequisite,  mnemonic,  mathemagenic  help,  representation  or  alternative  representation,  and  feedback.  The  idea  of 
presentation  forms  relating  to  different  categories  of  learning  appears  to  be  an  elaboration  of  Gagne’s  conceptions  of 
external  conditions  for  learning. 

Pinning  down  differences  and  similarities  between  Merrill's  CDT  and  Gagne’s  types  of  learning  represents 
something  of  a moving  target  because  both  systems  have  changed  over  the  years.  For  example,  Gagne's  “types  of 
learning”  evolved  into  a very  different  form  from  that  which  was  presented  in  1965.  In  general,  the  types  of  learning 


evolved  to  keep  up  with  changing  knowledge  about  learning  and  cognition  as  well  as  different  ideas  about  learning  from 
instruction  which  were  being  developed  by  contemporaries.  We  can  speculate  that  some  of  the  impetus  for  change  in 
Gagn6’s  types  of  learning  during  the  1970’s  may  have  come  from  ideas  such  as  Merrill’s  component-display  theory. 

C.M.  Reigeluth  developed  a model  for  instructional  design,  “the  “elaboration  theory,”  during  the  late  1970’s 
(Reigeluth,  1979).  As  an  extension  of  Merrill’s  component  display  theory,  elaboration  theory  may  be  seen  as  a 
“grandchild”  of  Gagne’s  seminal  work.  Elaboration  theory  takes  a broader  view  than  CDT  and  provides  guidance  for,  the 
design  of  instruction  for  complex,  unfamiliar,  multi-topic  content  rather  than  prescribing  the  form  of  encounter  for 
individual  lessons.  The  conditions-based  nature  of  the  model  is  seen  in  Reigeluth's  specification  of  three  differing 
structures,  conceptual,  procedural,  or  theoretical,  which  are  selected  based  upon  the  goals  of  the  course.  Later 
development  by  Reigeluth  includes  the  “simplifying  conditions  model”  which  retains  a cortditions-based  orientation  by 
suggesting  that  different  simplifying  conditions  structures  need  to  bq  developed  for  each  of  the  kinds  of  knowledge 
structures  described  (Reigeluth  & Rogers,  1980;  Reigeluth,  1992). 

In  part  an  extension  of  CDT,  Merrill  and  associates  have  formulated  a model  for  instructional  design  ,which  also 
has  links  to  Gagnd’s  work,  entitled  “ID^”  - a “second-generation”  instructional  design  model  (Merrill,  Li,  and 
Jones,  1990a,  1990b).  In  large  measure,  ID^  was  developed  to  assist  in  the  development  of  an  expert  system  for 
instructional  design,  “ID  Expert.”  ID^  vigorously  extends  the  basic  conditions  model,  making  more  explicit  the 
theorized  relationship  between  learning  outcomes  and  intemal/extemal  conditions  of  learning: 

a)  A given  learned  performance  results  from  a given  organized  and  elaborated  cognitive  structure, -which  we  will  call 
a mental  model.  Different  learning  outcomes  require  different  types  of  mental  models;  b)  the  construction  of  a 
mental  model  by  a learner  is  facilitated  by  instruction  that  explicitly  organizes  and  elaborates  the  knowledge  being 
taught,  during  the  instruction;  c)  there  are  different  organizations  and  elaborations  of  knowledge  required  to  promote 
different  learning  outcomes.  (Merrill,  Li,  & Jones,  1990b,  p.  8)  f 

Smith  and  Ragan  ( 1993)  developed  an  approach  to  the  design  of  instruction  that  exemplifies  and  elaborates  Gagn6’s 
theory.  Using  Gagne’s  types  of  learning,  they  postulated  a generalized  cognitive  process  necessary  for  the  acquisition  of 
each  of  the  different  learning  capabilities,  thereby  deriving  a system  of  instructional  strategy  recommendations  for 
different  types  of  learning.  Smith  and  Ragan  also  suggested  that  the  events  of  instruction  as  Gagne  portrayed  them 
insufficiently  considered  learner-generated  and  learner-initiated  learning,  and  restated  the  events  so  that  they  could  readily 
be  perceived  as  either  learner-supplied,  in  the  form  of  learning  strategies,  or  instruction-supported,  in  the  form  of 
instructional  strategies. 

Smith  and  Ragan  have  proposed  a model-Comparison  of  Generative/  Supplantive  Strategy  (COGSS)-  for 
determining  the  balance  between  instructional  strategies  (instruction-supplied  events)  and  learning  strategies  (learner- 
supplied  events)  based  upon  context,  learner,  and  task  variables.  They  also  proposed  that  there  is  a "middle  ground" 
between  instruction  supplied,  supplantive  (also  know  as  "mathemagenic")  events  and  learner-initiated  events,  in  which 
the  instruction  facilitates  or  prompts  the  learner  to  provide  the  cognitive  processing  necessary  to  an  instructional  event. 

Gagne's  instructional  theory  has  spawned  at  least  two  generations  of  instructional  design  .theories  that  have 
concretized,  elaborated,  and  exemplified  Gagnd's  conditions-based  propositions.  However,  his  influence  has  extended 
beyond  instructional  design  theory  into  other  areas  of  educational  design,  including  curriculum  design. 

Influences  of  Gagne’s  Theory  on  Curriculum  Design 

Although  Gagnd’s  work  was  directed  at  the  study  of  instruction,  not  at  teaching  or  curriculum,  a substantial 
influence  from  his  ideas  can  be  observed  in  those  fields.  Sometimes  harboring  conceptions  fundamentally  hostile  to  ideas 
grounded  in  learning  theory,  the  curriculum  and  teaching  methods  traditions  represent  a “tough  audience”  for  Gagne. 
However,  as  early  as  1966  W.B.  Ragan,  a curriculum  theorist  whose  Mqdern  Elementary  Curriculum  was  an  influential 
text  for  more  than  25  years,  made  extensive  use  Gagnd's  ideas  in  its  explanation  of  learning.  A survey  of  more  recent 
curriculum  texts  found  some  evidence  of  impact  of  Gagnes  ideas,  such  as  the  importance  of  consideration  of  types  of 
learning  when  determining  an  instructional  approach.  For  example, 'Pratt  (1980)  prescribed  the  matching  of  instruction 
with  objectives,  classifying  objectives  as  knowledge,  skills,  physical  development,  dispositions,  and  experiences.  With 


the  exception  of  "experiences,”  Pratt’s  categories  of  objectives  are  a close  fit  to  Gagnd’s  types  of  learning.  Indeed,  in 
Pratt's  chapter  describing  these  objective  categories,  he  cited  Gagne  and  provided  information  on  intellectual  skills  from 
Gagn6  and  Briggs  (1979).  The  basic  approach  that  Pratt  recommended,  deriving  plans  for  the  form  of  instruction  from 
(among  other  sources)  the  different  demands  placed  on  learners  in  achieving  different  sorts  of  objectives,  is  a solid 
application  of  Gagne’s  thinking. 

Posner  and  Rudnitsky  (1994)  proposed  a learning  task  categorization  scheme  very  similar  to  Gagnd’s: 

Employing  understandings  (cognitive  and  affective),  and  skills  (cognitive  skills,  psychomotor-perceptual  skills,  affective 
skills).  They  cited  Gagn6  as  one  of  the  sources  of  their  categorization  scheme.  In  another  text,  Robinson,  Ross,  and 
White  (1985)  identified  different  “growth  schemes”  for  different  types  of  learning:  inquiry  skills,  knowledge  outcomes, 
affective  outcomes.  This  idea  of  tying  different  educational  experiences  to  different  types  of  learning  is  very  much  in  the 
Gagn6  tradition. 

Too  often,  unfortunately,  curriculum  theorists  badly  misinterpret  Gagne’s  ideas  or  represent  early  work  as  if  it 
reflects  his  current  thinking.  In  citing  a 1967  definition  of  curriculum  offered  by  Gagnd,  Tanner  and  Tanner  (1980)  noted 
that  Gagne’s  definition  “assumes  that  learning  is  mechanical  and  linear,  and  that  the  learner  is  a mere  mechanism  to  be 
conditioned  toward  making  the  right  automatic  responses.”  (p.  26).  The  quote  to  which  Tanner  and  Tanner  are  responding 
, is  within  a discussion  of  cumulative  learning  effect  from  mastery  of  prerequisite  learnings,  “curriculum  is  a sequence  of 
content  units  arranged  in  such  a way  that  the  learning  of  each  unit  may  be  accomplished  as  a single  act,  provided  that  the 
capabilities  described  a specified  prior  units  (in  the  sequence)  have  already  been  mastered  by  the  learner.”  (Gagne,  1967,  p. 
23).  Perhaps  in  retribution  for  such  a simplistic  definition  of  curriculum.  Tanner  and  Tanner  provided  a characterization 
of  Gagnd’s  approach  as  being  based  on  conditioning  and  “right  responses.”  This  interpretation  is  totally  inaccurate.  In 
addition,  it  is  dismaying  that  in  writing  a 1980  text  that  the  authors  chose  to  refer  to  a 1967  publication  of  Gagnd's 
work,  rather  than  to  refer  to  the  most  recent  edition  of  Conditions  of  Learning  (1977)  that  would  have  more  accurately 
reflected  Gagne's  position  at  the  time.1 

In  a more  friendly  discussion,  but  still  inappropriate  classification,  Gagne’s  work  was  characterized  as  “social 
behaviorist”  by  Schubert  (1986).  The  research  methodology  works  of  Campbell  and  Stanley  (1966)  and  Kerlinger  (1973) 
were  also  placed  in  this  category,  so  use  of  the  term  “behaviorist”  is  apparently  an  original  one,  referring  essentially  to 
any  “scientific”  approach  to  education.  Schubert’s  discussions  of  Gagnd’s  work  are  not  trivializing,  however,  and 
represent  a thoughtful  and  informed  application  from  a curriculum  perspective. 

Conclusion 

As  described  in  the  discussions  of  Gagne’s  major  theoretical  contributions,  Gagne  has  developed,  refined,  and 
extended  his  theory  over  time.  This  continuing  development  based  upon  new  theory  and  research,  distinguishes  him 
among  scholars  in  educational/instructional  psychology.  His  instructional  theory  fulfills  many  of  Snow  and  Swanson's 
criteria  that  were  presented  at  the  beginning  of  this  chapter  as  components  of  an  instructional  theory 

a)  description  of  desired  end  states  or  goals  of  instruction  --  Gagnd's  types  of  learning; 

b)  description  of  goal-relevant  initial  learner  states  — Gagne's  prerequisite  analysis  via  learning  hierarchies; 

c)  description  of  the  transition  processes  — Gagn6's  internal  conditions  of  learning; 


1 The  propensity  to  cite  "intellectually  dated,"  rather  than  available  and  more  recent  work  that  would  more  accurately  represent 
the  current  development  of  Gagne's  theory  is  distressingly  common  in  not  just  the  curriculum  theory  literature,  but  in  many 
sources  that  cite  his  work.  Another  ex  ^mple  of  such  a problem  is  fou.id  in  Bower  and  Hilgard's  learning  theory  text  (1981), 
which  states  that  Gagne  is  (note  present  ttnse)  an  associationist  and  cites  a 1970  edition  of  Conditions  of  Learning.  In  some 
cases  these  statements  and  dated  citations  may  be  results  of  simple  oversights,  sloppy  scholarship.  In  other  cases  they  appear 
to  be  intentional  in  order  to  misrepresent  Gagne's  work  and  position.  Such  distortions  are  reprehensible  and  violate  the  very 
core  of  what  constitutes  "good  scholarship."  These  misrepresentations  are  made  all  the  more  unjust  by  their  targeting  the 
work  of  a scholar  who  has  for  many  been  the  very  personification  of  a "life-long  scholar."  His  work  has  developed 
continually  throughout  his  professional  life  as  he  examined  his  position  and  responded  with  thoughtful  revisions  of  his  ideas. 
He  has  also  been  meticulous  in  his  citation  of  the  work  of  others.  In  addition,  such  misrepresentation  has  created  no  end  of 
difficulty  for  those  in  our  field,  as  communication  with  colleagues  and  students  who  have  read  such  misrepresentations  can  be 
difficult,  time-consuming,  and  even  embarrassing  to  some  who  are  involved. 
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d)  detailing  of  instructional  conditions  that  promote  this  transition  - Gagnd's  external  conditions  of  learning; 

e)  assessment  of  performance  and  instructional  effects  - Gagn6  and  Briggs  explication  of  appropriate  assessment 
technology. 

Gagnd  has  left  some  questions  for  future  researchers  and  theorists  to  work  out.  One  of  these  areas  is  in  the 
description  of  the  transition  processes  between  novice  and  experts.  Gagnd  has  carefully  defined  some  of  these  internal 
conditions,  particularly  prerequisite  knowledge  states.  It  is  left  to  others  to  delineate  these  processes  more  completely  for 
the  various  learning  types  through  careful  empirical  validation.  A second  area  is  the  clear  explication  of  the  relatio  . is 
between  required  internal  processes  and  external  conditions  of  learning.  Within  this  study  may  be  the  examination  Nf 
necessary  versus  sufficient  conditions  to  support  internal  processing.  A third  area  of  extension  might  be  a further 
examination  of  the  relationship  between  declarative  knowledge  and  intellectual  skills.  There  appears  to  be  ample  evidence 
to  support  the  conclusion  that  the  ability  to  state  the  generalities  underlying  intellectual  skills  is  not  prerequisite  to  the 
learning  of  intellectual  skills.  However,  it  is  still  unclear  whether  some  other  aspect  of  the  declarative  nature  of 
intellectual  skills  might  be  prerequisite.  A test  of  the  robustness  of  a theory  is  not  only  the  number  of  questions  it 
answers,  but  also  the  number  of  questions  is  spawns.  Gagnd’s  instructional  theory  is  fertile  with  substance  to  be 
examined  by  future  scholars. 

It  is  difficult  to  overestimate  the  impact  R.M.  Gagnd  has  had  on  instructional  theory.  Although  his  has  not 
been  the  only  important  voice  in  shaping  the  field,  it  has  been  an  enormously  influential  one  by  virtue  of  the  prodigious 
volume  of  original  work  which  is  at  once  bold  in  its  conceptions  and  careful  in  scholarship.  This  combination  of 
thoroughly  grounded  yet  vigorously  inventive  work  has  left  a legacy  for  a field  of  study  to  build  upon. 
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THE  IMPACT  OF  QUALITATIVE  OBSERVATIONAL  METHODOLOGY  ON  THE  AUTHENTIC  ASSESSMENT 

PROCESS 


Introduction 

What  is  taught,  how  it  is  taught,  and  how  it  is  assessed  are  critical  issues  within  the  educational  community 
today.  The  concerns  that  students  in  our  American  educational  system  are  failing  to  learn  critical  thinking,  problem- 
solving, and  reasoning  skills,  and  do  not  have  the  ability  to  transfer  these  skills  to  real-world  situations  have  become  the 
focus  of  our  national  educational  reform  agenda.  In  order  to  deal  with  these  concerns,  educators  and  psychologists  have 
been  researching  various  methods  of  both  improving  student  learning  and  assessing  student  performance. 

In  order  to  acquire  accurate  and  useful  information  concerning  student  performance,  teachers  are  turning  away  from 
traditional  summative  evaluation  methods  toward  a formative  method  of  assessment.  This  method  of  assessment  is 
variously  known  as  authentic  assessment , alternative  assessment , and  performance-based  assessment.  These  terms  are 
often  used  interchangeably,  however  they  are  not  the  same.  According  to  Marzano,  Pickering,  & McTighe  (1993) 
authentic  assessment,  popularized  by  Grant  Wiggins  (1989),  conveys  the  idea  that  assessment  should  engage  students  in 
applying  knowledge  and  skills  in  the  same  way  they  are  used  in  the  ‘real  world’  outside  of  school  (p.  13).  Alternative 
assessment  applies  to  the  variety  of  assessments  that  differ  from  the  multiple-choice,  timed,  one-shot  approaches 
characterized  by  most  traditional  standardized  and  classroom  evaluation  methods.  Finally,  performance-based  assessment, 
according  to  Marzano,  Pickering,  & McTighe  (1993)  is  a broad  term,  encompassing  many  of  the  characteristics  of  both 
authentic  assessment  and  alternative  assessment  (p.  13).  Regardless  of  which  definition  is  used,  formative  assessment 
methods  require  the  assessor  to  employ  heightened  observational  skills  to  gather  careful  and  descriptive  assessment  data 
on  student  performance. 

Observation  is  an  important  method  of  data  gathering  in  authentic  assessment,  just  as  it  is  in  qualitative  research. 
The  process  of  data  gathering  for  the  sake  of  scientific  research  has  been  traditionally  thought  of  as  the  job  of  a full-time 
professional  researcher.  Today,  however,  “...  research  is  conducted  by  many  people  in  many  settings”  (Marriam,  1995, . 
p.l).  Page  and  Thomas  (1977)  defined  research  as  a “...systematic  investigation  to  increase  knowledge  and/or 
understanding”  (p.290).  One  method  of  systematic  qualitative  research,  referred  to  as  a case  study,  “is  an  intensive 
description  and  analysis  of  a phenomenon  or  social  unit  such  as  an  individual,  group,  institution,  or  community”  ” 
(Marriam,  1995,  p.  108).  Similarly,  during  an  authentic  assessment  process  the  classroom  teacher  is  involved  in 
intensive  description  and  analysis  of  a student’s  performance.  Classroom  teachers  use  anecdotal  records,  checklists,  logs, 
journals,  portfolios  and  interviews  as  authentic  assessment  data  gathering  tactics  in  much  the  same  way  that  qualitative 
researchers  gather  data  in  the  field.  In  a case  study  methodology,  for  instance,  the  primary  aim  of  the  systematic 
investigation  is  to  describe  the  observed  situation  “in  depth,  in  detail,  in  context,  and  holistically”  (Patton,  1987,  p.  19). 
This  is  also  the  goal  of  a classroom  teacher  engaged  in  the  authentic  assessment  process.  Parelleling  the  teacher’s  role 
in  the  classroom  with  that  of  a qualitative  researcher  places  a unique  perspective  on  the  traditional  view  of  a classroom 
teacher.  This  unique  perspective  expands  the  definition  of  today’s  classroom  teachers  to  that  of  a “teacher/researcher”, 
involved  in  documenting  and  evaluating  key  observable  educational  situations.  This,  in  turn,  results  in  the  creation  of  a 
broader,  more  genuine  picture  of  student  performance  and  offers  more  opportunities  for  the  improvement  of  student 
learning. 

The  use  of  qualitative  observational  methodology  is  a key  to  increasing  the  effectiveness  of  the  authentic 
assessment  process.  Through  consistent  and  systematic  classroom  observation,  the  teacher  is  able  to  gain  a deeper 
understanding  of  how  a child  learns  as  well  as  the  end  product  of  the  learning  experience.  Just  as  in  qualitative 
methodology,  the  “observer  is  the  instrument”  (Patton,  1987,  p.  12)  who  carefully  examines  trends  or  patterns  while 
emphasizing  “the  importance  of  the  meaning  of  human  behavior  and  the  social-cultural  context  of  social  interaction”  (et 
al.,  1987,  p.20).  The  integration  of  qualitative  observational  methods  can  provide  teachers  with  a systematic  pedagogical 
framework  in  which  to  improve  the  accuracy,  validity,  and  reliability  of  the  authentic  assessment  process. 

This  paper  focuses  on  a systematic  method  of  developing  an  authentic  assessment  instrument  that  incorporates  a 
three  phase  process  of  what  is  to  be  assessed,  how  it  is  to  be  measured,  and  how  to  interpret  verbal  and  non-verbal 
messages  or  data. 

What  is  to  be  assessed? 

Prior  to  the  implementation  of  any  form  of  assessment,  a clear  understanding  of  what  is  to  be  assessed  is 
necessary.  This  includes  a fundamental  understanding  of  the  purpose  of  the  assessment  process,  the  learner,  and  the 
subject  matter  to  be  taught. 
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If  the  primary  purpose  of  the  assessment  is  to  “describe  the  extent  to  which  students  have  attained  particular 
knowledge  and  skills,  your  assessment  should  focus  on  the  outcomes  or  product  of  student  learning”  (Herman, 

Aschbacher,  Winters,  1992,  p.23).  “However,  if  your  purpose  is  diagnosis  and  improvement,  such  as  diagnosing  a 
student’s  strengths  and  weaknesses,  prescribing  the  most  appropriate  instructional  programs,  or  identifying  strategies 
students  use  well  and  those  they  need  help  with,  you’ll  want  an  assessment  that  gives  you  information  about  the  process 
as  well  as  the  outcome”  (et  ah,  1992,  p.23). 

Understanding  and  recognizing  the  uniqueness  of  learners  and  their  learning  styles  is  an  important  step  in  the 
assessment  process.  The  multiple  intelligence  theory,  ad  vanced  by  Howard  Gardner  (1983),  espoused  a shift  in 
philosophy  in  which  the  assessor  recognizes  the  learner’s  framework  cf  intelligence  based  on  a broad  range  of  abilities. 
Fully  exploring  and  supporting  the  potential  of  all  learners  has  the  potential  to  maximize  the  learning  experience  and 
establish  a powerful  sense  of  self-confidence  for  all  learners. 

Finally,  a clear  understanding  of  the  subject  matter  is  a necessary  step  in  the  assessment  process.  Without  a clear 
understanding  of  the  subject  matter,  determining  important  learning  outcomes  would  be  a difficult,  if  not  impossible, 
task. 

How  is  it  to  be  measured? 

One  method  of  measuring  student  progress  is  through  the  use  of  a rubric.  A rubric  is  a rule  of  procedure.  “When 
the  word  is  used  in  connection  with  assessment,  a rubric  is  a scoring  guide  that  differentiates. ..on  an  articulated  scale...” 
(Jasmine,  1993.  p.9).  There  are  two  kinds  of  rubrics,  holistic  and  analytic.  According  to  Jasmine  (1993),  a holistic 
rubric  is  used  to  measure  the  overall  effect  of  a task  and  is  qualitative  in  nature.  An  analytic  rubric  consists  of  score 
points  assigned  to  various  elements  and  is  quantitative  in  nature.  Determining  clear  and  educationally  sound  learning 
outcomes  is  a major  step  in  the  assessment  process.  Well-constructed  learning  outcomes  establish  a strong  foundation 
for  prioritizing  and  creating  a properly  formulated  rubric. 

Setting  criteria  and  standards  of  a rubric  is  a main  component  of  authentic  assessment.  “Perhaps  most 
important,  scoring  criteria  make  public  what  is  being  judged  and,  in  many  cases,  the  standards  for  acceptable  performance. 
Thus,  criteria  communicate  your  goals  and  achievement  standards”  (Herman,  Aschbacher,  Winters,  1992,  p.44). 

Selecting  the  task  to  match  the  authentic  assessment  is  also  a crucial  point  in  the  assessment  process.  Aligning 
the  assessment  task  with  the  intended  outcomes  should  be  done  to  ensure  that  the  assessment  can,  in  fact,  measure  the 
learning  to  take  place.  “When  considering  assessment  tasks,  your  best  choices  are  those  you  believe  most  closely  target 
your  instructional  aims  and  allow  your  students  to  demonstrate  their  progress  and  capabilities”  (Herman,  Aschbacher, 
Winters,  1992,  p.33).  Herman,  Aschbacher,  Winters  (1992)  posed  six  questions  to  help  educators  choose  good  tasks: 

• Does  the  task  match  specific  instructional  intentions? 

• Does  the  task  adequately  represent  the  content  and  skills  you  expect  students  to  attain? 

• Does  the  task  enable  students  to  demonstrate  their  progress  and  capabilities? 

• Does  the  assessment  use  authentic,  real-world  tasks? 

• Does  the  task  lend  itself  to  an  interdisciplinary  approach? 

• Can  the  task  be  structured  to  provide  measures  of  several  goals?  (p.35-37) 

Choosing  an  appropriate  task  must  thenPbe  followed  by  describing  the  assessment  task  in  a manner  that  is  clear  and 
documentable  in  order  for  others  to  replicate  and/or  interpret  the  finding.  Finally,  Herman,  Aschbacher,  Winters  (1993) 
offer  a set  of  criteria  that  help  to  ensure  that  the  appropriate  learning  tasks  lead  to  sound  assessment.  These  criteria  are: 

• Do  the  tasks  match  the  important  outcome  goals  you  have  set  for  the  students? 

• Do  they  pose  an  enduring  problem  type,  the  type  of  problems  and  situations  that  students  are  likely  to  face 
repeatedly  in  school  and  their  future  lives? 

• Are  the  tasks  fair  and  free  of  bias? 

• Will  the  tasks  be  credible  to  important  constituencies? 

• Will  the  tasks  be  meaningful  and  engaging  to  students  so  that  they  will  be  motivated  to  show  their  capabilities? 

• Are  the  tasks  instructionally  related/teachable? 

• Are  the  tasks  feasible  for  implementation  in  your  classroom  or  school  in  terms  of  space,  equipment,  time,  costs,  and 
so  forth?,  (p.  41-43) 


How  to  interpret  verbal  & non-verbal  messages  or  data. 

The  assessment  process  is  a matter  of  the  verification  of  student  learning.  In  the  authentic  assessment  process, 
students  are  engaged  in  authentic  and  meaningful  tasks.  According  to  Wiggins  (1993),  the  assessment  process  is 
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authentic  when  we  directly  examine  student  performance  on  worthy  intellectual  tasks  (p.  3)  therefore,  the  reliance  on 
sound  first-hand  observation  of  both  verbal  and  non-verbal  learning  situations  is  a vital  component  in  authentic 
assessment.  Teachers  involved  in  authentic  assessment  need  to  develop  strong  observational  skills.  Harp  (1991) 
discusses  observation  as  a powerful  tool  for  the  evaluation  of  students  as  they  are  engaged  in  the  performance  of  real 
world  tasks  (p.  75). 

First-hand  observation  provides  an  understanding  of  both  the  activity  and  the  context  in  which  the  activity  is 
performed.  It  allows  the  observer  to  make  more  reliable  inferences  concerning  the  performance  activity.  Better 
understanding  and  more  reliable  inferences  are  a direct  result  of  the  observer’s  ability  to  concentrate  on  worthy  intellectual 
tasks  in  order  to  evaluate  the  performance  activity  more  cleariy. 

The  ability  to  interpret  verbal  and  non-verbal  messages  accurately,  validly,  ar  d reliably  through  the  observation 
of  learning  situations  requires  particularly  careful  analysis  and  judgment  on  the  part  of  the  observer.  “The  validity  and 
reliability  of  qualitative  data  depend  to  a great  extent  on  the  methodological  skill,  sensitivity,  and  training  of  the 
evaluator.  Systematic  and  rigorous  observation  involves  far  more  than  just  being  present  and  looking  around”  (Patton, 
1987,  p.  8).  The  value  of  observational  data  is  developed  through  the  observer’s  descriptive  documentation  of  the  setting, 
the  activities,  and  meaning  of  what  was  observed. 


Critical  Viewing 

The  teacher  has  always  been  a critic  to  some  extent,  but  are  teachers  critical  viewers  as  well?  According  to  James 
Brown  (1991),  "The  critic,  and  the  critical  viewer,  must  be  grounded  in  two  essential  areas  of  this  act  of  critical 
judgment:  the  area  of  facts  (what  is  informational  data)  and  the  area  of  norms  or  standards  (what  ought  to  be  criteria)"  (p. 
24).  Thus,  teachers  are  critical  viewers  by  nature,  but  perhaps  not  by  systematic  process. 

The  process  of  critically  viewing  performances  involving  human  action  has  not  been  adequately  addressed  through 
research.  Perhaps  this  is  because  no  critical  viewing  model  exists  to  guide  teachers  in  systematically  assessing  their 
students.  Traditionally,  critical  viewing  has  involved  the  interpretation  of  verbal  and  non-verbal  information  involving 
some  form  of  media  such  as  movies,  photographs  or  television.  As  Ploghoft  and  Anderson  (1982)  stated,  "individuals 
trained  in  critical  viewing  skills  will  be  equipped  with  criteria  for  evaluating  intention,  motives,  and  audience  response 
(students),  for  assigning  value  or  worth  to  the  message"  (p.  5).  This  paper  attempts  to  provide  a critical  viewing  model 
for  education  by  providing  teachers  with  a systematic  three  phase  process  for  the  development  of  an  authentic  assessment 
instrument  which  includes  a critical  viewing  component.  However,  before  discussing  the  third  phase  that  involves 
critical  viewing  for  teacher/researchers,  it  is  important  to  understand  how  and  where  critical  viewing  skills  developed. 

Critical  reading  skills  evolved  into  critical  viewing  skills  in  order  to  better  analyze  electronically  mediated 
messages.  As  John  Long  (1989)  observed,  "much  of  the  groundwork  for  critical  viewing  was  derived  from  work  in 
critical  reading.  In  critical  reading,  Individuals  learn  procedures  to  extract  and  assimilate  from  the  print  media"  (p.  12). 
Critical  viewing  follows  a similar  process  but  involves  electronic  media.  As  society  and  media  became  more 
sophisticated,  the  need  to  understand  how  information  was  interpreted  through  electronic  media  grew. 

Several  projects  and  studies  were  conducted  to  analyze  and  develop  standard  critical  viewing  skills  for  elementary, 
middle  school,  high  school  and  adult  students.  The  elementary  project  was  conducted  by  the  Southwest  Educational 
Development  Laboratory,  WNET  a public  television  station  in  New  York  published  the  middle  school  segment,  the  Far 
West  Laboratory  in  San  Francisco  developed  the  high  school  study  and  Boston  University's  School  of  Public 
Communications  developed  the  adult  program.  The  project's  goal  was  to  assist  individuals  in  monitoring  their  own 
viewing  habits;  in  other  words,  to  teach  people  to  view  television  from  a perspective  other  than  that  of  the  medium. 

There  have  been  many  different  sets  of  skills  developed  for  various  mediated  situations  and  targeted  at  certain  age 
levels.  For  example,  the  Southwest  Educational  Development  Laboratory  defined  critical  viewing  skills  as  the  ability  to 
distinguish  program  elements,  make  judicious  use  of  viewing  time,  understand  the  psychological  implications  of 
advertising,  distinguish  fact  from  fiction,  recognize  and  appreciate  differing  views,  understand  content  of  dramatic 
presentation,  understand  style  of  dramatic  presentation  and  understand  the  relationship  between  television  and  the  printed 
word  (Brown,  1991,  p.96).  Although  a general  standard  of  critical  viewing  skills  is  difficult  to  identify,  the  majority  of 
projects  and  studies  share  the  concept  that  people  must  not  be  passive  consumers  of  information  because  they  lose  the 
ability  to  understand  the  world  from  varied  perspectives.  Adams  & Hamm  (1988)  observed,  "Intelligent  consumers  of 
video  information  need  to  be  able  to  sort  out  the  meaningful  from  the  trivial"  (p.  82).  An  increased  level  of  critical 
awareness  has  the  potential  to  move  the  viewer  from  passive  information  processing  to  active  information  processing. 
This  is  true  of  any  viewing  situation,  be  it  mediated  or  live.  In  the  age  of  information,  how  one  views  life  can  help  or 
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hinder  understanding  reality.  Thus,  critical  viewing  <s  more  than  understanding  mediated  messages.  It  is  understanding 
all  types  of  messages  from  mediated  to  live.  As  Adams  & Hamm  stated,  "Effective  viewing  of  real  world  images  requires 
a critical  sense  for  both  information  and  shades  of  meaning"  (1988,  p.  82). 

Critical  Viewing  in  an  Educational  Setting 

Critical  viewing  of  student  performance  is  the  next  step  in  the  evolution  of  critical  viewing  skills.  The  general 
framework  consists  of  "the  principles  of  intervention,  goal  attainment,  cultural  understanding,  and  literacy"  (Long,  1989, 
p.  13).  Intervention  involves  understanding  the  process  of  perception  and  may  be  altered  through  self-regulatory  skills. 
This  is  a major  step  because  it  is  at  this  level  that  the  viewer  stops  being  a passive  viewer  and  becomes  an  active  viewer. 
The  active  viewer  begins  to  understand  how  his/her  own  internal  perceptions  and  biases  are  developed.  Individuals  bring  a 
set  of  beliefs  to  every  situation.  Discovering  how  to  separate  one’s  own  perception  from  the  visual  messages  in  a 
situation  is  the  key  to  becoming  a critical  viewer.  An  example  of  this  is  a teacher  using  a rubric  to  assess  a child’s 
performance.  In  order  to  incorporate  critical  viewing  into  the  assessment,  the  teacher  must  use  intervention  to  understand 
how  his/her  internal  perspectives  may  affect  the  assessment.  For  instance,  if  a teacher  is  tired  or  has  had  difficulty  with  a 
particular  child  in  the  past,  a skewed  perspective  of  the  child's  ability  may  occur.  This  skewed  perspective  may  in  turn 
alter  the  assessment  process. 

The  next  step  in  the  process  of  becoming  a critical  viewer  is  goal  attainment.  Goal  attainment  involves 
monitoring  and  understanding  one’s  own  reasons  for  attending  to  the  learning  situation  as  well  as  recognizing  the 
student’s  own  (and  possibly  different)  rationale  for  participation.  In  other  words,  the  viewer  and  the  student  performing 
the  learning  activity  each  have  a set  of  goals.  These  goals  do  not  necessarily  refer  to  learning  activity  goals  but  refer  to 
the  personal  goals  of  the  teacher  and  of  the  student  at  the  time  of  assessment.  The  teacher  may  want  the  student  to 
complete  the  activity  in  a certain  way  and  the  student  may  want  to  just  get  through  the  task.  It  is  the  responsibility  of 
the  critical  viewer  to  understand  his/her  personal  goals  as  well  as  the  goals  of  the  person  being  assessed.  By 
accomplishing  this,  the  critical  viewer  can  ensure  that  the  goals  of  the  learning  activity  are  mutually  understood  and  that 
the  evaluation  of  the  students'  performance  is  not  adversely  affected  by  a clash  of  personal  goals. 

The  third  step  in  the  critical  viewing  process  is  cultural  understanding.  Cultural  understanding  is  knowing  where 
the  learning  situation  exists  within  the  social  schema.  It  is  the  opposite  of  the  intervention  step  because  the  viewer  now 
focuses  on  external  awareness  of  the  perceptions  and  biases  of  the  individuals  being  assessed.  For  example,  students' 
race,  religion,  gender,  etc.  play  a large  role  in  their  perception  of  any  learning  situation.  The  students’  perceptions  and 
biases  will  be  likely  to  affect  their  performances  and  thus  adversely  alter  the  assessment  unless  the  teacher/viewer 
incorporates  cultural  context  into  the  assessment  situation. 

The  final  step  in  the  critical  viewing  process  is  literacy.  Literacy  is  understanding  the  overall  grammar  of  the 
learning  situation,  such  as  body  language  or  actions  of  the  learner  in  the  course  of  an  activity  (physical,  behavioral, 
and/or  cognitive).  It  is  at  this  step  in  the  process  that  the  viewer  can  take  all  of  the  knowledge  gained  from  the  first  three 
steps  of  the  critical  viewing  process  and  actually  interpret  what  is  seen  during  the  learning  activity.  For  example,  the 
body  language,  vocal  tone  and  verbal  usage  of  a diabetic  child  with  low  blood  sugar  will  appear  as  a normal  child 
behaving  poorly.  A trained  and  well-prepared  teacher  critically  viewing  the  learning  situation  will  assess  the  whole  child 
within  the  learning  context.  A teacher  passively  viewing  the  situation  might  assess  the  child  incorrectly  by  not  being 
aware  of  perceptions  and  biases,  personal  and  other  goals,  external  awareness  of  others'  perceptions  and  biases,  and  the 
literacy  of  how  to  communicate. 

The  four  steps  to  critical  viewing  skills  create  skilled  observers  who  are  trained  and  prepared  for  authentic 
assessment.  Critical  viewing  adds  an  important  component  to  the  authentic  assessment  process,  filling  in  valuable 
contextual  information  about  performance,  thus  creating  a more  valid  and  reliable  assessment. 

Pre-Assessment  Focusing  Session 

A well-trained  and  prepared  critical  viewer  recognizes  the  need  for  intense  focus  and  concentration  in  the 
observation  process.  Prior  to  entering  an  observational  situation  for  the  purposes  of  assessment,  a classroom  teacher 
must  set  aside  time  to  mentally  prepare  for  the  complex  exercise  of  interpreting  the  verbal  and  non-verbal  messages  in  a 
student’s  performance.  This  process  is  referred  to  in  this  paper  as  a pre-assessment  focusing  session.  A pre-assessment 
focusing  session  utilizes  the  four  steps  of  critical  viewing  and  is  intended  to  focus  a teacher/researcher’s  critical  viewing 
skills  in  order  to  filter  out  personal  perceptions  or  biases  that  could  be  present  during  an  assessment  situation.  This 
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process  involves  little  time  commentment  but  such  focused  preparation  is  essential  for  effective  critical  viewing  and 
assessment.  Once  a teacher/researcher  becomes  familar  with  the  steps  of  critical  viewing  the  pre-assessment  focusing 
session  should  become  an  integral  part  of  the  assessment  process. 

Summary 

What  is  taught,  how  it  is  taught,  and  how  it  is  assessed  are  critical  issues  within  the  educational  community. 

The  three  phases  discussed  in  this  paper  integrate  the  major  steps  involved  in  the  development  of  a sound  authentic 
assessment  instrument. 

The  first  phase  addresses  the  need  to  carefully  formulate  a clear  understanding  of  What  is  to  be  assessed?  This 
includes  the  identification  of  the  purpose  of  the  assessment,  recognizing  the  uniqueness  of  the  learners,  and  possessing  a 
knowledge  base  in  the  relevant  subject  matter. 

The  second  phase  looks  closely  at  How  is  it  to  be  measured?  Determining  appropriate  learning  outcomes  will 
drive  the  instructional  and  assessment  processes,  therefore  the  formulation  of  the  learning  outcomes  and  the  criteria  and 
standards  that  will  be  applied  to  them  are  critical  components.  This  leads  directly  to  the  selection  of  a suitable 
assessment  task.  The  assessment  task  is  an  essential  component  of  authentic  assessment  and  is  not  only  important  from 
the  stand  point  of  validation  of  student  learning  but  is  vital  for  the  students  themselves  to  understand  the  expectations  of 
the  learning  task.  Ralph  Tyler  (1949),  father  of  modem  evaluation,  commented  that  without  an  understanding  of  what  a 
learner  is  supposed  to  know  or  how  it  is  to  be  expressed,  makes  determining  whether  or  not  learning  has  actually  occurred 
very  difficult. 

The  third  phase  describes  how  to  interpret  verbal  and  non-verbal  messages  or  data  in  the  observational  process  of 
authentic  assessment  Goodman  stated,  “teachers  are  constant  kid  watchers"  (1986,  p.41).  As  such,  classroom  teachers 
need  to  be  trained  and  well-prepared  observers  of  all  activities  throughout  the  school  day.  Teachers  must  remain  focused 
and  objective  just  as  qualitative  researchers  in  scientific  fields  do.  Trained  and  well-prepared  classroom  teachers  become 
more  astute  observers  of  the  underlying  meaning  and  social  context  of  student  behavior,  thus  creating  the  potential  for 
improved  quality  of  their  findings  and  ultimately  of  their  assessment  procedures. 

Since  teacher  preparation  is  essential  for  accurate  interpretation  of  verbal  and  non-verbal  messages,  the  inclusion 
of  a brief  pre-assessment  focusing  session  prior  to  an  observational  assessment  will  positively  impact  the  assessment 
process.  This  focusing  session  sharpens  a teacher’s  concentration  and  heightens  the  teacher’s  awareness  of  personal 
interventions,  goals,  the  cultural  context,  and  literacy  issues  that  may  interfere  with,  contaminate  or  unfairly  influence 
the  assessment  process. 

The  integration  of  the  three  phases  process  described  in  this  paper  into  the  authentic  assessment  process  provides 
the  classroom  teacher  with  a systematic  method  of  developing  an  effective  assessment  instrument.  Processing 
information  through  the  first,  second  and  the  third  phases  gives  a classroom  teacher  a sense  of  confidence  in  the 
development  of  their  own  authentic  assessment  instrument  as  well  as  their  observational  skills. 

As  classroom  teachers  strengthen  their  observational  skills  and  begin  to  recognize  their  roles  as  teacher- researchers 
involved  in  important  qualitative  educational  research,  hidden  benefits  are  likely  to  be  discovered.  The  impact  of  these 
hidden  benefits  will  be  seen  on  two  levels.  On  one  level,  the  classroom  teacher  will  experience  an  increase  in  confidence 
and  professionalism.  On  another  level,  there  will  be  direct  positive  effect  on  daily  instructional  and  social  classroom 
activities.  The  significance  of  these  powerful  hidden  benefits  will,  by  their  very  nature,  have  a positive  impact  on 
teachers’  creativity  and  on  their  overall  effectiveness  in  the  instructional  and  assessment  process. 
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BACKGROUND 

As  the  use  of  computers  has  become  increasingly  common  in  society,  the  number  of  people  working  with  video 
display  terminals  has  risen  dramatically.  The  computer  operator,  possibly  the  weakest  link  in  the  computer's 
productivity,  has  been  frequently  overlooked  by  manufacturers  and  software  developers.  Human  engineering  and 
ergonomics  are  in  a relatively  primitive  state  compared  with  the  physical  sciences  that  created  the  equipment. 

One  important  aspect  of  this  interaction  between  the  operator  and  the  computer  is  the  area  of  screen  layout  and 
design,  and  a question  primary  to  good  design  is  that  of  the  color  configuration  of  the  screen  text  itself.  Color  has  been 
identified  as  being  an  important  design  consideration  (Dwyer,  1972),  but  variations  in  the  configuration  of  color  have 
received  less  conclusive  research  attention.  Taylor  and  Murch  (1986)  considered  both  text  and  graphic  displays  when 
reviewmg  the  principles  for  effective  color  coding  on  video  monitors.  They  caution  that  both  background  and  text  colors 
must  be  considered  and  that  the  relative  degree  of  saturation  is  a critical  factor.  The  concept  of  perceived  brightness  and 
saturation  of  color  has  been  researched  by  a number  of  investigators  with  sometimes  conflicting  and  contradictory  results 
(Stanton,  1992;  Lippert  1985;  Wald,  1967).  Joyner  (1992)  concluded  that  amber  text  on  green  was  the  most  effective  for 
reducing  errors  in  ducument  proofreading. 

Research  on  the  use  of  color  in  improving  the  usability  of  a screen  display  has  also  provided  conflicting  results. 
Kopala  (1981)  and  Sidorsky  (1982)  found  color  to  be  a positive  factor  while  Tull  is  (1981)  and  Christ  & Teicher  (1973) 
found  color  to  impair  performance.  Joyner  (1989)  found  keyboard  error  detection  unaffected  by  color.  Other  researchers 
have  investigated  the  effectiveness  of  variations  in  color  contrast,  again,  with  mixed  results  (Schnure,  1996;  Edstrom, 
1987;  Gruning,  1985;  Lalomia  & Happ,  1987).  Clearly,  the  often  contradictory  findings  of  these  studies  emphasize  the 
need  for  more  systematic  investigation  of  the  factor  of  screen  text  color  configuration,  particularly  with  regard  to  the 
characteristics  of  the  particular  user  and  user's  experience  with  a video  display  terminal. 

This  study  was  conducted  to  determine  the  effect  of  the  color  configuration  of  a video  display  terminal  (VDT)  on 
the  operator's  ability  to  detect  typographical  effors  in  keyboarded  copy.  Two  primary  research  questions  were  addressed: 

(1)  Is  there  an  interaction  between  keyboarding  experience  and  the  color  configuration  of  the  VDT  in  terms  of 
proofreading  ability,  and  (2)  Is  there  a relationship  between  the  color  configuration  of  the  VDT  and  the  location  of 
undetected  keyboarded  errors  on  the  screen. 

METHODS  AND  PROCEDURES 

The  research  sample  in  the  study  consisted  of  97  undergraduate  students  enrolled  in  a college  of  business  at  a 
Pennsylvana  university.  To  assure  an  accurate  representation  of  varied  ability  levels  and  ages,  participants  were  drawn 
from  two  regional  campuses  as  well  as  the  main  campus.  Subjects  were  subsequently  divided  into  3 ability  level  groups- 
-beginning,  intermediate,  and  advanced-based  upon  their  reported  keyboarding  experience. 

The  design  employed  in  this  study  was  a 3 x 4 repeated  measures  design  that  compar  d three  ability  levels  across 
four  color  configurations:  (1)  white  text  on  blue  background,  (2)  blue  text  on  white  background,  (3)  green  text  on  amber 
background,  and  (4)  amber  text  on  green  background.  Each  treatment  condition  reflected  comi  - <nly  used  color 
configurations  used  in  popular  word  processing  packages. 

The  materials  developed  for  the  study  consisted  of  four  business  documents  containing  similar  numbers  of 
typographical  and  layout  errors,  equally  distributed  in  all  four  quadrants  of  the  screen.  The  documents  were  created  using 
a popular  word  processing  package  and  were  drawn  from  a departmental  exemption  examination  used  in  a keyboarding  and 
document  formatting  class. 

Each  subject  was  presented  with  the  four  documents  in  a controlled  laboratory  environment  and  was  instructed  to 
locate  and  mark  the  errors  using  the  overstrike  feature  of  the  word  processing  package.  Ten  minutes  were  provided  for 
work  on  each  document.  All  documents  were  rotated  through  each  treatment  at  each  campus  location  to  ensure  that  every 
subject  received  all  treatment  conditions. 

FINDINGS  AND  RESULTS 

Using  the  tabulated  data,  an  analysis  of  variance  for  repeated  measures  was  conducted  on  the  treatments  by 
experience  levels  to  test  the  first  research  question:  Is  there  an  interaction  between  keyboarding  experience  and  color 
configuration  of  the  video  display  terminal  in  terms  of  proofreading  ability?  The  results  of  that  analysis  revealed  a 
significant  interaction  (F=3.504;  p=.0023).  See  Table  1. 
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Table  1. 

ANALYSIS  OF  VARIANCE  ON  TREATMENTS  BY  EXERIENCE  LEVELS 


Source 

df 

Sum  of 
Squares 

Mean 

Square 

F Value 

p Value 

Experience  Level  (A) 

2 

14.117 

7.058 

3.761 

.0268 

Color  Configuration  (B) 

3 

8.894 

2.965 

1.094 

.3518 

Experience  x Color  (AB) 

6 

56.951 

9.492 

3.504 

,0023 

Error 

282 

763.904 

2.709 

The  means  and  standard  deviations  for  all  color  confgurations  and  experience  levels  is  presented  in  table  2. 


Table  2 

TABLE  OF  MEANS  AND  STANDARD  DEVIATIONS 
FOR  COLOR  CONFIGURATION  BY  EXPERIENCE  LEVEL 


Color 

Configuration 

WhiteText/Blue 

Background 

Blue  Text7 
White  Background 

Mean 

SD 

Mean 

S.D. 

Advanced 

Level 

N=45 

4.111 

1.668 

4.444 

1.531 

Intermediate 

Level 

N=32 

3.406 

1.563 

3.156 

1.439 

Beginning 

Level 

N=20 

3.65 

1.461 

2.221 

2.221 

Totals 

N=97 

3.784 

1.609 

3.773 

1.765 

Table  2 (Continued) 


Color 

Configuation 

Green  Text/ 
Amber  Back 

ground 

Amber  Text/ 
Green  Background 

Totals 

Mean 

S.D. 

Mean 

S.D. 

N 

Mean 

S.D. 

Advanced 

Level 

N=45 

3.689 

1.635 

3.022 

1.215 

180 

3.817 

1.601 

Intermediate 

Level 

N=32 

3.469 

1.646 

3.5 

1.606 

128 

3.383 

1.553 

Beginning 

Level 

N=20 

3.3 

1.593 

4.25 

1.552 

80 

3.612 

1.747 

Totals 

N=97 

3.536 

1.621 

i 

3.433 

1.485 

388 

3.433 

1.63 

Pair-wise  comparisons  of  all  means  via  the  Fisher  Protected  Least  Significant  Differences  Test  (PLSD)  indicated 
that  for  advanced  level  subjects  either  white  text  or  blue  text  was  significantly  better  than  any  green  or  amber  formats.  In 
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contrast,  the  multiple  comparison  tests  indicated  that  beginning  level  subjects  performed  significantly  better  with  the 
amber  text  on  green.  Significant  comparisons  are  presented  in  Table  3. 


Table  3. 

SIGNIFICANT  POST-HOC  COMPARISONS  OF  TREATMENTS 
BY  LEVELS  OF  EXPERIENCE 


COMPARISON 

MEAN 

DIFFERENCE 

FISHER 

PLSD 

DECISION 

Adv  White/Blue  vs  Adv 

.1.089 

.656 

Adv  White/Blue  > Adv 

Amber/Green 

Amber/Green 

Adv  White/Blue  vs  Int 

.955 

.719 

Adv  White/Blue  > Int 

Blue/White 

Blue/White 

Adv  White/Blue  vs  Beg 

.861 

.836 

Adv  White/Blue  > Beg 

Blue/White 

Blue/White 

Adv  Blue/White  vs  Adv 

.756 

.656 

Adv  Blue/White  > Adv 

Green/ Amber 

Green/Amber 

Adv  Blue/White  vs  Adv 

1.422 

.656 

Adv  Blue/White  > Adv 

Amber/Gieen 

Amber/Green 

Adv  Blue/White  vs  Int 

1.038 

.719 

Adv  Blue/White  > Int 

White/Blue 

White/Blue 

Adv  Blue/White  vs  Int 

1.288 

.719 

Adv  Blue/White  > Int 

Blue/White 

Blue/White 

Adv  Blue/White  vs  Int 

.976 

.719 

Adv  Blue/White  > Int 

Green/Amber 

Green/Amber 

Adv  Blue/White  vs  Int 

.944 

.719 

Adv  Blue/White  > Int 

Amber/Green 

Amber/Green 

Adv  Blue/White  vs  Beg 

1.194 

.836 

Adv  Blue/White  > Beg 

Blue/White 

Blue/White 

Adv  Blue/White  vs  Beg 

1.144 

.836 

Adv  Blue/White  > Beg 

Green/Amber 

Green/Amber 

Adv  Green/ Amber  vs  Adv 

.667 

.656 

Adv  Green/ Amber  > Adv 

Amber/Green 

Amber/Green 

Adv  Amber/Green  vs  Beg 

1.228 

.836 

Adv  Amber/Green  < Beg 

Amber/Green 

Amber/Green 

Int  Blue/White  vs  Beg 

1.094 

.886 

Int  Blue/White  < Beg 

Amber/Green 

Amber/Green 

Beg  White/Blue  vs  Beg 

1.000 

.983 

Beg  White/Blue  < Beg 

' Amber/Green 

Amber/Green 

Since  a significant  interaction  was  found,  research  queston  1 is  answered  in  the  affimative.  A significant 
interaction  does  exist  between  keyboarding  experience  level  and  the  color  configuration  of  the  video  display  terminal  in 
terms  of  proofreading  ability.  The  interaction  data  is  displayed  graphically  in  Figure  1 . 

Analysis  of  data  strongly  suggest  that  experienced  keyboarders  will  demonstrate  greater  success  in  proofreading 
when  working  with  either  blue  or  white  text.  The  findings  of  several  other  researchers  have  indicated  that  dark  spectrum 
colors  make  better  background  choices  and  this  study  supports  that  conclusion  with  regard  to  the  use  of  white  text  on  a 
blue  background  with  experienced  keyboarders.  On  the  other  hand,  research  that  suggests  that  amber  text  on  green  is 
superior  may  only  apply  to  beginning  level  or  inexperienced  users. 
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4.1 

NUMBER  3 9 " 
CF 

ERRORS  3 7 " 
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Analysis  of  data  related  to  research  question  2:  Is  there  a relationship  between  the  color  configuration  of  the 
video  display  and  the  location  of  undetected  keyboarded  errors  on  the  screen,  was  addressed  via  analysis  of  variance  of  the 
mean  numbers  of  undetected  effors  in  each  of  the  four  screen  quadrants.  An  analysis  of  Quadrant  1 data  did  not  produce  a 
significant  effect  (F=l  .349,  p=.2587).  Similarly,  analysis  of  Quadrant  2 data  did  not  produce  a significant  effect 
(F=1.152,  p=  3286). 

The  analysis  of  Quadrant  3 data  did  reveal  a significant  treatment  effect  (F  = 3.433, p =.0175).  Further  analysis 
of  all  possible  pair-wise  comparisons  via  the  Fisher  Protected  Least  Significant  Difference  Test  (PLSD)  was  conducted. 
Significant  comparisons  are  presented  in  Table  4. 

Table  4. 

SIGNIFICANT  POST-HOC  COMPARISONS  OF  TREATMENTS 
ON  QUADRANT  3 ERROR  DATA 

(P  < 05)  


COMPARISON 

MEAN 

DIFFERENCE 

FISHER 

PLSD 

DECISION 

Green/ Amber  vs  White/Blue 

.206 

.167 

Green/ Amber  > White/Blue 

Blue/White  vs  White/Blue 

.247 

.167 

Blue/White  > White/Blue 

Significantly  fewer  undetected  errors  occurred  in  Quadrant  3 when  the  color  configuration  was  White/Blue  rather  than 
Green/Amber  or  Blue/White. 

An  analysis  of  the  data  related  to  quadrant  4 produced  a significant  treatment  effect  (F=7.343  p=.000I).  Further 
analysis  of  all  possible  pair-wise  comparisons  via  the  Fisher  Protected  Least  Significant  Difference  Test  (PLSD)  was 
conducted.  Significant  comparisons  are  presented  in  Table  5. 


Table  5. 

SIGNIFICANT  POST-HOC  COMPARISONS  OF  TREATMENTS 
ON  QUADRANT  4 ERROR  DATA 


COMPARISON 

MEAN 

DIFFERENCE  ' 

FISHER 

PLSD 

DECISION 

White/Blue  vs  Blue/White 

.227 

.151 

White/Blue  < Blue/White 

White/Blue  vs  Green/ Amber 

.278 

.151 

White/Blue  < Green/ Amber 

Blue/White  vs  Amber/Green 

.216 

.151 

Blue/White  > Amber/Green 

Green/ Amber  vs 
Amber/Green 

.268 

.151 

Green/Ambei>  Amber/Green 

These  comparisons  indicated  that  both  White/Blue  and  Amber/Green  formats  were  significantly  more  effective  in 
reducing  the  number  of  undetected  errors  in  the  fourth  quadrant.  Of  particular  interest  is  the  fact  that  both  of  these 
formats  consisted  of  light  text  on  a dark  background,  a fact  which  may  have  made  the  text  more  distinct  and  consequently 
more  effective. 

The  quadrant  data  also  suggests  that  the  greatest  number  of  unde  .ected  errors  occured  in  the  third  and  fourth 
(lower)  quadrants  of  the  screen.  This  may  be  explained  by  eye  movement  studies  which  demonstrate  that  the  least  time  is 
spent  viewing  this  area. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  conclusions  and  recommendations  that  follow  are  based  on  the  findings  of  this  study  and  may  serve  as 
guidelines  for  future  research. 

1 . Since  there  is  a significant  difference  in  the  proofreading  abilities  of  keyboarders  possessing  advanced  levels 
of  keyboarding  experience  when  different  color  configurations  are  used,  the  color  configuration  of  the  video  display 
terminal  is  critical.  The  most  effective  color  configuration  for  advanced  ability  keyboarders  appears  to  be  blue  text  on  a 
white  background.  Beginning  level  keyboarders  seem  to  work  most  effectively  when  using  amber  text  on  a green 
background.  No  particular  screen  configuration  seems  to  assist  keyboarders  with  intermediate  level  skills. 

2.  The  proofreading  ability  of  the  keyboarder  increases  in  proportion  to  his  or  her  experience  level.  Individuals 
with  the  most  keyboarding  experience  are  better  proofreaders. 

3.  The  location  of  the  error  within  a particular  quadrant  of  the  screen  will  determine  the  relative  probability  with 
which  the  error  is  detected  by  the  operator.  The  results  indicate  that  more  errors  go  undetected  in  the  lower  half  of  the 
screen  (Quadrants  3 & 4)  than  in  the  upper  half  of  the  screen  (Quadrants  I & 2).  These  results  are  in  agreement  with 
findings  of  other  researchers  who  have  shown  that  the  eye  moves  from  the  upper  left  comer  of  the  display  and  scans  from 
left  to  right,  starting  at  the  top.  This  visual  scanning  follows  the  same  general  ordered  pattern  as  that  used  in  reading 
printed  text.  Thus  the  results  have  direct  implications  for  the  instructor  as  proofreading  techniques  are  taught 

4.  White/Blue  and  Amber/Green  configurations  appear  to  be  most  effective  in  helping  users  detect  errors  in  the 
lower  two  quadrants.  These  both  represent  light  text  on  dark  backgrounds  which  may  make  the  text  more  distinct.  It 
would  appear  that  particular  attention  should  be  given  to  making  text  in  these  quadrants  as  distinctive  as  possible  to 
compensate  for  this  apparent  disparity  in  color  effectiveness. 
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ReBoot's  scripts  are  deftly  written  to  be  easily  understood  by  children,  with  a wink  and  a nod  to  the  adult  viewers. 

The  citizens  of  Mainframe  speak  in  a high  tech  dialect  that  incorporates  computer  terms  that  have  crept  into  the 
everyday  language  of  the  automated  age.  Bob  complains  when  a task  is  not  his  'function'.  Enzo  is  encouraged  to  'cut 
and  paste*  the  truth.  When  Enzo  gets  discouraged.  Dot  counsels  him  not  to  'quit  file'  so  easily. 

(Excerpt  form  [ReBoot]  Press  Kit  - Alliance  Communications  and  BLT  Productions) 

. . . we  need  to  look  not  just  at  work  that  collectivities  collaboratively  do  to  construct  gendered  worlds  but  also  look  at 
the  work  that  language  does  to  limit,  shape,  make  possible,  one  kind  of  world  or  another.  . . .The  individual  subject 
is  understood  at  one  and  the  same  time  to  be  constituted  through  social  structures  and  through  language,  and  becomes 
a speaking  subject,  one  who  can  continue  to  speak/write  into  existence  those  same  structures  through  those  same 
discourses.  But  as  a speaking  subject,  they  can  invent,  invert  and  break  old  structures  and  patterns  and  discourses  and 
thus  speak/write  into  existence  other  ways  of  being. 

(Bronwyn  Davies) 

Introduction 

The  genesis  of  this  paper  developed  through  an  awareness  of  the  growing  number  of  television  programs  that  feature 
information  technologies  like  computers  as  significant  narrative  devices.  These  devices  include  the  use  of  computer  based 
technologies  as  environments  in  which  characters  interact  (e.g.  virtual  worlds)  and  the  use  of  computers  as  tools  that 
characters  utilize  to  confront  conflict  (e.g.  computers  as  problem  solving  tools).  Characters  in  these  programs  are  often 
themselves  part  human,  part  machine;  cyborgs,  fused  and  bonded  with  technology.  These  television  programs  include 
science  fiction  shows  (e.g.  Star  Trek:  The  Next  Generation,  Star  Trek:  Deep  Space  Nine,  Star  Trek:  Explorer,  Space: 
Above  and  Beyond,  Babylon  5,  Sea  Quest,  Dangerous  Games,  etc.)  commercials  (e.g.  IBM,  Microsoft,  Intel,  Apple,  etc.) 
and  children's  shows  (e.g.  Power  Rangers,  ReBoot,  Johnny  Quest,  VR  Troopers,  Reality  Check,  etc.).  In  addition  to  their 
use  as  narrative  devices  in  these  programs,  computers  are  increasingly  made  reference  to  and  talked  about  on  numerous 
prime  time  situation  comedies  (e.g.  Home  Improvement).  Within  the  state-of-the-art  worlds  of  television  programs  using 
high  tech  innovations  are  embedded  representations,  including  the  use  of  language,  which  inscribe  simplified, 
compartmentalized  and  encased  conceptions  of  race,  gender,  and  power.  Characters  in  these  programs  engage  in  a pseudo- 
language appropriated  from  computers  and  science,  where  a specialized  vocabulary  is  part  of  the  “natural”  lexicon. 

As  a student  and  teacher  of  educational  technology  watching  these  programs  based  on  themes  derived  from  high  tech 
innovations,  I am  concerned  with  how  young  viewers  through  the  constructed  character  identities  in  these  programs 
transform  difference  in  relation  to  the  technical  language  and  encased  representations  of  gender,  race,  and  power  to  a 
potentially  alienating  otherness  in  young  people.  A Saturday  morning  children's  cartoon  like  “ReBoot”  appears  in  40 
countries  throughout  the  world.  How  do  young  people  who  do  not  possess  the  same  literacies  and  competencies  as  these 
television  characters  apparently  possess,  interpret  this  jargonized  language  and  imagery?  Will  some  young  people  confuse 
their  differences  with  deficiency?  How  will  this  structure  their  relationships  with  technology  in  using  the  computer  as  a 
tool  for  learning?  How  may  young  people  be  relegated  to  low  tech  ghettos  of  impoverished  knowledges  about 
technology,  computers,  and  computer  culture  in  the  way  these  shows  through  their  imagery,  language  and  narrative 
structure  help  construct  these  ghettos  and  position  the  low  tech  viewer  in  relation  to  the  cultures  advanced  in  these 
shows? 

These  questions  evolved  while  watching  programs  like  “Star  Trek:  Voyager”  and  “ReBoot”  with  the  four  young 
people  with  which  I live  [ages  9 (male),  12  (female),  13  (male),  and  15  (female)].  Even  though  these  young  people 
posses  a working  knowledge  of  computers  and  have  access  to  computers  both  at  school  and  at  home,  aspects  of  these 
shows  including  stereotypical  descriptions  of  race  and  gender  do  not  resonate  within  their  experience.  They  see  Western 
stereotypical  representations  of  race  and  they  do  not  match  their  experiences  with  their  friends  of  color  - they're 
experiences  are  more  idiosyncratic  and  unique  compared  to  the  offered  stereotypes.  They  view  the  tight  fitting  uniforms 
that  the  slim  female  characters  in  these  shows  often  wear  and  the  girls  do  not  see  an  ideal  of  femaleness,  instead  they 
often  see  unrealistic  and  undesirable  conceptions  of  physical  beauty. 

The  question  kept  surfacing,  “In  what  ways  do  ‘different’  children  (i.e.  young  people  with  different  social  positioning 
and  literacies  then  those  depicted  by  the  characters  in  these  programs)  attach  meanings  to  the  words  and  metaphors, 
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characters  and  narratives  of  these  TV  programs  and  how  di  .he  meanings  they  attach  impact  their  relationship  with 
themselves  and  the  societies  and  communities  in  which  they  must  negotiate  their  lives?” 

Examining  the  role  of  popular  culture  in  framing  issues  of  race,  gender,  and  agency  including  the  power  of  television 
with  its  capacities  for  representation  and  construction  of  meaning,  is  central  to  understanding  how  the  mass  media 
inform,  structure,  and  constrain  the  daily  lives  of  many  young  people  in  postmodern  societies(Agger,  1992).  Through 
portrayals  of  postmodern  life,  television  programs  teach  specific  ways  of  knowing  and  being  about  race,  culture,  and 
gender  (Giroux,  1989;  Cortds,  1995).  In  discussing  ways  that  high  tech  shows  may  position  themselves  in  relation  to 
young  viewers,  I would  like  to  focus  on  the  processes  with  which  these  shows  simplify,  compartmentalize,  compress, 
and  encase  language,  personal  agency  and  identity.  I will  frame  this  analysis  through  an  examination  of  an  episode  of  a 
popular  Saturday  morning  cartoon,  “ReBoot* ’. 

As  Bronwyn  Davies  (1993)  observes  language  and  discourse  play  a constitutive  role  in  the  construction  of  gender. 

The  languages  with  which  young  people  engage  themselves,  others,  and  television  programs  have  consequences  on  the 
ways  they  situate  their  identities  of  race  and  gender,  as  well  as  personal  and  social  agency.  Language  limits  and  shapes, 
making  one  world  in  which  ‘difference*  is  viewed  as  ‘other*  more  possible  than  another  where  ‘difference*  is  viewed  as 
part  of  a larger  collective.  Young  people  watching  television  and  interpreting  language  spoken  by  television  characters  are 
forms  of  discourse,  albeit  largely  unidirectional  discourses  from  program  to  viewer(s).  The  types  of  languages  which  are 
spoken  on  television  programs  based  on  themes  derived  from  high  tech  innovations  are  by  and  large  unnegotiated:  young 
people  must  construct  meanings  within  themselves  or  perhaps  between  themselves  and  the  others  they  may  be  watching 
with  without  an  exchange  with  the  characters.  They  cannot  say,  “What  did  you  mean  that  you  ’downloaded  files'  or  'How 
are  you  using  the  word  ‘bitmap*?**  Young  television  viewers  cannot  interact  with  characters  in  a cartoon  who  use  a great 
deal  of  computer  terminology  or  directly  challenge  their  use  of  language  or  portrayals  of  gender  or  racial  stereotypes.  The 
discourses  do  not  extend  directly  back  to  the  media.  This  hermetic  encasement  of  specialized  language  associated  with 
computers  has  the  potential  to  alienate  young  people  and  separate  them  from  their  sense  of  personal  agency.  This 
separation  positions  the  young  viewer  as  an  “other**  in  relation  to  dominant  forms  of  language  about  computers  and  shifts 
the  responsibility  to  the  viewer  to  understand  and  make  sense  of  what  is  seen  and  heard.  The  ’pedagogy’,  if  you  will,  of 
television  is  similar  to  the  dominant  model  of  education  which  positions  students  as  passive  receptacles  of  information. 

Related  to  the  way  that  television  encases  language  is  the  way  that  language  has  developed  around  computers  in 
which  a similar  reification  of  meaning  exists.  Through  increasing  interaction  with  computers  at  home,  school,  and  work 
computer  terminology  has  crept  into  the  popular  culture  and  is  often  used  as  metaphors  to  describe  aspects  of  daily  lives. 

A glance  at  the  comic  section  of  any  newspaper  will  reveal  numerous  references  to  computer  culture.  Computers  have 
served  as  the  dominant  metaphor  of  the  human  mind  for  the  cognitive  sciences  for  decades.  In  discussing  the  growing 
infusion  of  computer  terminology  as  a form  of  metaphor  into  language  Sardello  (1985)  writes,  “Computer  terminology 
is  certainly  not  a living  language,  but  rather  the  enslaving  language  of  turning  every  form  of  speech  into  an  object  to  be 
manipulated  by  the  totalitarian  grammar  of  computational  logic”. 

Living  in  western  postmodern  societies  engages  young  people  on  multiple  and  fluid  levels  of  social  interactions 
where  experience  is  fractured  though  an  ever-present  interaction  with  mass  media  including  television,  film,  radio,  and 
computers.  By  simplifying  and  compartmentalizing  complex  issues  and  relationships,  television  tries  to  neatly  encase  the 
unknowable  and  constantly  changing  through  a process  of  fixing  and  rendering  static  and  objectified  that  which  is  always 
changing.  An  example  of  these  processes  in  television  is  the  always  neatly  drawn  line  between  good/evil  and  right/wrong 
in  many  programs.  This  simplified  encasement  of  meaning  through  dualism  goes  counter  to  the  complexities  of  western 
postmodern  life.  Developing  a practice  to  challenge  the  ways  that  these  programs  manage  and  control  the  potential  for 
‘difference’  in  multivalent  and  ambivalent  characters,  meanings,  and  interpretations  becomes  an  important  task  and 
challenge  for  educators.  By  challenging  dominant  cultures’  needs  for  legitimization,  consumerism,  and  productivism,  the 
literature  surrounding  critical  pedagogy  which  began  with  the  writings  of  Paulo  Friere  (1970)  has  provided  an  educational 
framework  with  which  to  address  issues  arising  from  critical  cultural  studies.  Critical  pedagogy  provides  the  point  of 
praxis,  the  meeting  of  theory  and  practice  through  the  political. 

Critical  Pedagogy 

Ideas  of  critical  and  emancipatory  pedagogy  provide  a framework  with  which  to  form  strategies  of  practices  to  work 
with  young  people  to  reinterpret  the  potentially  qlicnating  messages  of  the  programs  discussed.  By  framing  pedagogy  as  a 
cultural  practice  of  cultural  production,  critical  pedagogy  addresses  the  relationships  between  language,  power,  knowledge 
and  individual  agency  and  views  learning  as  part  of  the  process  of  social  change  itself  (Kanpol,  1994).  Always  viewing 
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teaching  as  a political  act,  critical  pedagogy  strives  to  understand  how  power  works  within  particular  historical,  social, 
and  cultural  contexts  in  order  to  engage  and,  when  necessary  change  such  contexts.  Henry  Giroux  (1994)  writes: 

By  refuting  the  objectivity  of  knowledge  and  asserting  the  partiality  of  all  forms  of  pedagogical  authority,  critical 
pedagogy  initiates  an  inquiry  into  the  relationship  between  the  form  and  content  of  various  pedagogical  sites  and  the 
authority  they  legitimate  in  securing  particular  cultural  practices. 

If  one  frames  viewing  television  as  an  act  of  cultural  learning  then  a practice  influenced  by  critical  pedagogy  may 
help  challenge  static  representations  of  language,  gender,  race,  and  personal  agency.  Critical  pedagogy  opens  new 
discourses  with  popular  culture  texts.  For  example,  while  discussing  the  technical  language  used  in  a particular  television 
show  with  a young  person  a teacher  may  ask  her  or  him  questions  about  dominant  forms  of  language,  consumerism,  and 
their  relationship  to  these  dominant  discourses.  Questions  about  the  young  person’s  conceptions  of  what  types  of  people 
might  better  understand  this  technical  language  may  prove  productive?  As  Apple  (1990)  challer.ges  teachers  must  enable 
students  to  “..inquire  as  to  why  a particular  collectivity  exists,  how  it  is  maintained  and  who  benefits  from  it.”  Questions 
within  a critical  framework  help  the  young  person  to  side-step  being  positioned  against  these  dominant  discourses  as 
‘other’  and  powerless,  and  help  to  redefine  notions  of  agency  and  power. 

Theorists  working  within  this  framework  often  draw  upon  poststructural  ideas  to  gain  entrance  into  the  power 
struggles  of  representations  of  race  and  gender  in  popular  cultures.  Poststructural  theory  defies  universalizing  meanings 
and  provides  pedagogical  theorists  a means  to  break  notions  of  totalizing  and  totalizable  power  and  control.  In 
poststructuralism  power  is  seen  as  dispersed,  as  networked  and  horizontal  rather  than  vertical  and  concentrated. 
Poststructuralism  rewrites  notions  of  power  (away  from  the  dualisms  of  powerful/powerless;  dominant/subordinate)  and 
toward  notions  of  lines  and  vectors  of  power  that  can  be  concentrated  within  nodes  of  institutional  practice,  and  within 
particular  historical  contexts  and  situations.  Poststructuralism  assert  that  language  is  an  important  mode  of  representation 
that  constitutes  what  counts  as  reality.  In  this  sense  students  can  become  what  Henry  Giroux  calls  “border  intellectuals”  - 
crossing  the  boundaries  of  dominant  power  structures  including  the  politics  of  representation  that  television  structures. 

Influenced  by  poststructural  ideas  a teacher  thinks  of  ways  of  helping  young  people  rewrite  the  potentially  encasing 
ideas  of  gender,  race,  and  personal  agency  of  television  programs.  This  work  can  happen  in  many  forms  including 
discussion,  writing,  and  the  creation  of  media  including  music,  photography,  and  video  (Paley,  1995).  Important  to 
cultural  and  pedagogical  theorists  is  the  politicization  of  culture.  These  ideas  often  considered  liberal  and  even  radical  are 
ignored  by  many  theorists  in  education  who  refuse  to  sec  school  as  a site  of  political  struggle  for  identity.  Yet  as  Toni 
Morrison  (1993)  notes,  “Excising  the  political  from  the  life  of  the  mind  is  a sacrifice  that  has  proven  costly.” 

Some  who  have  examined  the  literature  on  critical  pedagogy  contend  that  these  theories  are  highly  abstract  and  may 
not  necessarily  sustain  the  daily  practice  of  the  education  its  supporters  advocate  (Ellsworth,  1989).  Despite  extensive 
literature  in  this  area,  scholars  rarely  in  their  writings  locate  their  theoretical  constructs  within  actual  localized  practice.  If 
one  is  to  believe  the  arguments  of  postructuralism,  that  meaning  is  negotiated  and  constructed  by  the  individual  within  a 
social  context , then  it  is  important  for  us  as  educators  to  find  local  and  personal  venues  to  realize  critical  theories.  This 
should  not  be  interpreted  as  a pedagogy  that  leads  to  isolated  individualism  but  to  a pedagogy  that  helps  the  individual 
situate  their  place  in  a living  history.  Michael  Shapiro  (1995)  writes  the  significance  of  emphasizing  individual  lives: 

...is  not  its  refocusing  from,  collectivities  to  individuals  but  its  identification  of  the  plurality  of  consequences, 
registered  in  different  lives,  that  become  available  when  descriptions  of  the  intersections  of  different  flows  of 
activities  are  allowed  to  displace  the  official  language  of  the  order. 

Theory  only  exists  in  the  way  that  it  is  acted  upon.  A variety  of  educators  drawing  upon  different  aspects  of  critical 
pedagogy  have  found  ways  of  deconstructing  some  of  the  hegemonic  influences  within  postmodern  western  societies 
including  the  struggles  to  situate  multicultural  activism  in  the  University  (Ellsworth,  1989)  politicize  power  in  teacher 
education  (Giroux,  1994),  bring  critical  readings  of  mass  media  into  the  high  school  classroom  (Fehlmand,  1992)  and 
find  new  ways  of  re-reading  and  re-writing  ‘history’  through  testimony  (Felman  & Laub,  1992). 

If  one  views  education  as  a social  and  political  process  and  wishes  to  support  the  individual  student’s  construction  of 
meaning  then  it  becomes  necessary  to  find  local  venues  of  practice  to  address  power  and  agency  -to  irritate  the  taken  for 
granted  simplification,  compartmentalization  and  rigid  encasement  of  representations  of  power,  race,  and  gender.  Critical 
pedagogy  supported  by  poststructural  ideas  provides  ways  of  finding  the  “interpretive  breaks”;  the  points  at  which  we  get 
close  enough  to  sec  the  cracks  in  an  ideology,  power  struciure,  or  representation  by  making  visible  the  invisible  walls 
that  maintain  inequities.  These  walls  could  be  a subtext  or  the  words  or  actions  that  go  unspoken,  intentionally  or  not. 
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I turn  now  to  a specific  examination  of  a cartoon  to  highlight  some  of  the  ideas  that  I have  advanced  about  high  tech 
television  shows  and  their  capacity  to  simplify,  compartmentalize,  compress,  and  encase  language,  gender,  race,  personal 
agency  and  identity. 

“ReBttrt!! 

“ReBoot”  is  an  ABC  Saturday  morning  cartoon  that  airs  in  40  countries  in  North  and  South  America,  South  Africa, 
and  western  Europe.  The  show  is  translated  from  English  into  several  languages  including  Spanish,  French,  and  German. 
“ReBoot”  takes  place  on  an  “Island”  called  “Mainframe”  within  a computerized  world  . All  characters  in  ReBoot  are 
computer  programs  called  “Sprites”.  I chose  to  analyze  “ReBoot”  for  *hr  paper  because  of  its  use  of  computers  as  the 
axis  around  which  the  entire  narrative  rotates.  Characters  in  the  cartoon  use  language  that  is  specific  to  computers  in  the 
real  world  to  describe  more  universal  human  actions  and  characteristics  (i.e.  improvements  in  a character  may  be  described 
as  an  “upgrade”).  There  are  many  parallels  in  language  that  a young  person  who  uses  a computer  regularly  may  make 
with  her  or  his  own  life.  Yet  there  are  many  who  do  not  possess  this  type  of  ‘literacy’.  One  wonders  of  the  meanings  that 
will  be  constructed  by  a young  person  who  feels  alienated  by  such  language. 

There  are  five  main  characters  in  the  cartoon.  “Bob”  is  a ‘Defender,’  a guardian  program  from  the  advanced 
civilization  of  the  Super  Computer.  Using  Glitch,  a multi-functional  computerized  protection  device  he  wears  on  his 
wrist.  Bob  represents  the  young  male  warrior  ready  to  protect  Mainframe  from  any  electronic  threat.  “Dot  Matrix”  is  the 
shows  female  character  and  as  the  shows  press  release  states  is  “an  attractive  entrepreneur”.  Dot’s  primary  business 
venture  is  as  owner  of  “Dot’s  diner”.  “Enzo”,  Dot's  adolescent  brother,  serves  as  the  character  young  viewers  may  most 
associate  with  - often  too  young  to  be  in  the  action,  he  must  wait  on  the  sidelines  while  Bob  and  Dot  do  the  work  of 
adults  - saving  Mainframe.  Enzo  is  portrayed  as  the  ’typical'  American  adolescent  boy  with  crooked  baseball  cap, 
skateboard,  and  dog,  “Frisket”.  Positioned  against  this  representation  of  boyness  how  would  a young  viewer  in  South 
America  construct  his  or  her  conception  of  ‘boyness’  or  ‘girlness’?  One  never  hears  of  parents  nor  sees  Dot  and  Enzo’s 
parents  in  the  show.  Through  their  speaking  of  standard  American  English,  these  characters  display  typical  European 
characteristics  of  whiteness.  “Phong”,  a cyborg  character , half  machinelike,  half  human,  is  the  only  identifiable  character 
of  color  - golden  colored  and  displaying  the  west's  stereotypical  images  often  associated  with  mature  Asian  men  including 
goatee  and  a steady  stream  of  Confucian  wisdom.  Phong  serves  as  Mainframes  spiritual  mentor  and  the  wise  teacher  of 
Bob,  Dot  and  Enzo. 

“Megabyte”,  a computer  virus  programmed  to  always  grow  and  seek  more  power,  is  the  story's  darkly  colored 
villain.  During  most  episodes  he  aspires  to  increase  his  foothold  in  Mainframe  which  includes  controlling  about  a third 
of  the  city.  But,  of  course,  he  always  fails  and  is  made  to  look  foolish  in  the  process  - his  greed  for  power  blinding.  I 
wonder  if  Megabyte  displays  in  this  television  program  what  Morrison  (1992)  calls  in  American  literary  tradition  the 
“choked  representation  of  an  Africanist  presence.”  Hexadecimal  portrayed  as  a female  is  the  shows  other  primary  villain,  a 
computer  virus  written  to  create  chaos  not  seek  power.  Her  face  never  moves  when  she  speaks  instead  she  puts  on  a series 
of  masks  to  portray  her  mood  or  display  a feeling  she  is  trying  to  convey.  Like  Dot,  Hexadecimal’s  hourglass  figure 
conforms  to  the  western  stereotype  of  an  attractive  women. 

At  some  point  during  each  episode  a purple  gaseous  cube  descends  from  the  sky  and  covers  a sector  of  Mainframe. 
This  is  a game  being  loaded  by  the  “user”  the  operator  of  the  computer  in  which  Mainframe  is  located.  No  one  is  certain 
of  the  “user's”  identity,  they  only  know  her  or  him  as  the  entity  that  tests  the  Sprite’s  (usually  Bob,  Dot,  and  Enzo) 
abilities  to  fight  and  win  through  physical  strength  and  technical  ability.  The  games  are  often  “Dungeons  and  Dragons” 
styles  games  which  are  generally  violent  role  playing  games,  mostly  played  in  real  life  by  adolescent  boys.  These  games 
often  include  medieval  contexts  inhabited  by  wizards,  fighters,  thieves,  and  damsels  in  distress.  Games  also  take  the  form 
of  professional  basketball  games,  wars,  and  car  races.  How  will  young  viewers  who  are  not  familiar  with  the  conventions 
of  these  games  and  scenarios  map  their  own  experience  onto  their  viewing  of  ReBoot?  When  the  sprites  enter  a game 
they  must  “ReBoot”  : a process  where  they  are  assigned  a “character”  which  defines  their  skills  and  weapons  in  the  game. 
Passing  through  a series  of  levels,  the  defenders  must  win  each  game  against  the  “user”  or  else  the  sector  will  go  ‘offline’ 
turning  each  of  the  sprites  in  that  sector  into  “Nulls”  energy  eating  slugs  that  live  in  the  lower  levels  of  Mainframe. 

In  addition  to  the  main  characters  is  a sub  population  called  “Binoms”  who  are  more  machinelike  and  look  more  like 
children's  toys.  They  are  generally  shaped  in  the  form  of  l’s  and  0’s.  These  characters  are  unidimensicnal  and  possess 
stereotypical  personalities  (e.g.  a television  with  hands  that  is  the  “typical”  French  waiter,  complete  with  mustache, 
exaggerated  accent,  and  towel  over  the  arm).  Binoms  passively  populate  Mainframe  as  it’s  proletariat  - workers  who  do 
word  processing,  graphics  and  manipulate  numbers,  giving  the  main  characters  a populated  context  in  which  to  act. 


762 


776 


The  episode  examined  for  this  paper  is  entitled  “Identity  Crisis”,  a two  part  story  in  which  Dot  tries  to  free  a section 
(Sector)  of  Mainframe  under  Megabyte’s  control  by  importing  the  PID  (personal  identification  numbers)  of  every  Binom 
in  the  sector  into  a computer.  These  PID  number  are  important  because  an  individual,  including  a Defender,  without  one 
falls  under  the  computer-based  control  of  Megabyte.  A PID  is  equivalent  to  personal  freedom.  Importing  these  numbers 
into  a computer  allows  Phong  to  convert  the  sector  into  an  Energy  Park,  a haven  free  from  the  dominating  control  of 
Megabyte.  They  must  give  up  their  only  source  of  personal  freedom  to  obtain  the  greater  freedom  of  their  community.  In 
the  episode  after  an  emotional  plea  from  Dot,  the  Binoms,  rather  passively  give  up  their  PIDs. 

Notions  of  “revolution”  are  seen  as  mass,  largely  passive  efforts  carried  out  by  a few  vanguard  leaders  and  not 
decentered  individual  (situated)  actions.  Through  this  vision,  masses  of  undifferentiated  binoms  give  up  what  makes 
them  individual  for  ’freedom’,  an  interesting  metaphor  for  passive  reception  of  mass  culture.  Agency  is  handed  over  to  the 
few  and  powerful  protectors  of  freedom.  In  “ReBoot”  freedom  means  not  under  the  control  of  the  malevolent  Megabyte 

In  this  episode  when  it  comes  time  to  give  up  the  PID  numbers  and  all  the  Binoms  begin  to  release  theirs  codes  en 
masse  there  is  a medium  shot  of  a Binom  family.  The  father  in  the  family  not  only  releases  his  own  PID  but  the  PID  of 
the  mother  and  the  baby  without  getting  their  consent.  Agency  is  usurped  from  the  female  and  child  by  the  father. 

Dot,  with  her  distorted  Barbiesque  physique,  is  the  mastermind  of  this  “sector  conversion”.  It  is  through  her  careful 
planning,  using  a computer,  that  this  grand  event  is  possible.  She  prepares  for  her  final  meeting  with  the  Binoms  to 
collect  their  PIDs  by  changing  from  her  tight  fitting  space-age  pantsuit  to  a blue  tight  fitting  business  suit  with  black 
and  white  top,  gold  accessories,  and  a short  black  miniskirt.  The  outfit  is  completed  with  a pair  of  round  black  eyeglasses 
that  she  never  wears  in  the  series  until  this  point.  What  do  the  glasses  signify?  That  she  knows  better,  but  needs  to 
reinforce  this  with  the  Binoms. 

The  creators  of  the  show  assume  the  viewers  posses  a tremendous  amount  of  technical  language  frequently  using 
computer  words  as  everyday  parts  of  language  (See  Table  1). 


Table  1 

Examples  of  technical  terminology  used  in  ReBoot 


glitch 

nullify 

bug 

alphanumeric 

sprite 

modemed 

floating  point 

bitmap 

8-bit 

overwrite 

low  density 

dot  matrix 

backspace 

wait  state 

encryption 

hidden  file 

COMMAND.COM 

algorithm 

unformat 

downloading 

What  kinds  of  knowledge  and  skills  must  one  have  in  order  to  have  agency  in  Mainframe?  It  is  clear  from  the  success 
of  sprites  like  Bob,  Dot,  and  Enzo  that  one  must  be  extremely  computer  literate  as  well  as  knowing  how  to  fight. 

Physical  dexterity  and  strength  are  valued  and  crucial  to  win  a game  and  defeat  the  plots  of  Megabyte  and  Hexadeciamal . 

Conclusion 

There  is  not  one  way  to  read  media.  Any  single,  particular  strategy  of  interpretation  leaves  out  other  ways  of  seeing. 
Whenever  an  interpretation  is  imposed  on  a perspective,  alternative  interpretations  are  repressed.  Because  no  reading  is 
innocent,  the  questions  that  go  unposed  about  a text  and  what  it  means  are  as  important  as  the  questions  that  are  asked. 
Learning  is  always  situated  and  knowledge  is  always  incomplete.  As  Ellsworth  and  Miller  (1996)  write: 

...constructing  and  disrupting  fixed  meanings  of  difference  is  profoundly  situational,  and  often  tedious.  It  also'is 
personal  and  social  at  the  same  time,  risky,  never  predictable,  and  requires  imagination  and  courage  of  the  intellect  as 
well  as  of  the  I cart. 

An  idea,  theory,  power  structure  only  exists  in  the  way  that  it  is  acted  upon  not  on  the  way  that  it  is  theorized  about. 
Ultimately  what  is  ti  e good  of  situating  the  analysis  of  the  “ReBoot”  episode  within  critical  pedagogy?  There  can  be 
advantages  to  illumir  o*ing  what  goes  unspoken  in  a piece  of  media,  but  for  what  purpose  and  for  whom?  Without  praxis 
the  re-creation  of  knowledge  is  incomplete. 

For  my  part  I can  situate  my  analysis  with  the  four  young  people  with  whom  I live.  I can  ■ k them  questions  about 
their  understanding  of  how  language  is  used  to  construct  meanings  of  race,  gender,  and  agency  ,nd  frame  issues  like  how 
a female  character  displays  stereotypical  characteristics  of  femalcness  through  her  actions.  I have  done  this  and  often  am 
met  with  confusion,  yet  creating  confusion  can  open  new  ways  of  seeing.  Together  we  have  tried  to  re-construct 
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meanings  about  various  representations  in  “ReBoot”.  For  example,  I have  asked  them  other  ways  that  a character  could 
confront  conflict  instead  of  using  a particular  technology.  Discussions  of  negotiation  and  compromise  have  developed 
from  this  question  as  well  as  the  problematic  nature  of  the  conflicts  themselves  (i.e.  what  is  presented  as  important 
enough  to  engage  the  characters). 

I can  also  challenge  the  teachers  that  I work  with  while  teaching  them  video  production  to  find  ways  in  their  own 
practice  with  students  to  rupture  the  inscriptions  of  identity  presented  by  television  and  other  mass  media.  These 
challenges  will  be  highly  idiosyncratic  and  unique  to  each  teacher  as  will  the  way  that  each  teacher  responds  '.o  these 
challenges.  Through  video  production,  teachers,  together  with  their  students,  can  take  stereotypes  and  encased 
representations  of  language,  race,  and  gender  and  turn  them  back  on  themselves  to  rupture  meanings  and  make  room  for 
new  representations. 

Ultimately  what  becomes  clear  is  that  one  cannot  proscribe  this  kind  of  cultural  work  - it  is  important  to  situate 
action  within  a specific  community  at  a specific  time.  Regardless  of  whether  representations  are  accepted  passively  or 
actively  challenged,  meanings  will  always  be  assigned  to  representations. 
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Abstract.  This  case  study  involved  designing  an  electronic-based  environment  to  explore  the  effectiveness  of 
electronic  mail  (e-mail)  as  a writing  tool  for  dialogue  journaling.  The  setting  for  this  study  was  in  an  intermediate  level 
reading  and  writing  class  in  the  American  English  Institute  Program  on  the  campus  of  a large  public  university.  Over  a 
period  of  nine  weeks,  six  random  assigned  English  as  a second  language  (ESL)  students  in  the  class  wrote  dialogue 
journals  to  their  instructor  using  e-mail  while  the  rest  of  the  students  in  the  class  wrote  dialogue  journals  to  the 
instructor  using  paper  and  pencil.  The  issues  investigated  were:  What  were  the  students*  attitudes  toward  dialogue 
journal  writing  via  e-mail?  What  was  the  instructor’s  perception  regarding  e-mail  dialogue  journal  writing?  What 
problems  occurred  in  the  process  of  using  e-mail  as  a tool  for  doing  dialogue  journal  writing?  In  what  ways,  were  e-mail 
journals  different  from  paper  journals?  The  findings  of  the  study  show  that  a variety  of  fe^iOi , combined  to  exert  an 
influence  on  the  participants*  attitudes  towards  e-mail.  Limited  knowledge  about  e-mail  system  prevented  some  students 
from  taking  a full  advantage  of  e-mail  as  a unique  communication  tool.  Comparison  of  e-mail  journals  and  paper 
journals  reveals  that  e-mail  created  a different  writing  mode  than  that  of  paper  and  pencil. 

Literature  Review 

A dialogue  journal  is  a daily  written  communication  between  two  persons.  In  the  classroom  setting,  the  teacher 
and  the  student  are  the  two  partners  involved  in  this  written  conversation.  Each  day,  the  student  puts  an  entry  into  a 
bound  notebook  and  hands  in  the  norebook  to  the  teacher.  Students  are  not  assigned  a topic  to  write  about.  They  can 
write  about  anything  of  personal  interest  and  concern.  Instead  of  grading  or  correcting  the  student's  writings,  the  teacher 
writes  an  individual  response  to  the  student's  entry.  The  next  day,  the  teacher  gives  the  notebook  back  to  the  student  so 
that  the  interchange  can  continue.  The  written  exchange  can  develop  into  a year-long  daily  conversation  between  the 
teacher  and  the  student. 

Since  its  first  discovery  as  a classroom  practice  in  1979,  dialogue  journaling  has  been  widely  used  in  writing 
classrooms  of  English  as  a second  language  (ESL)  to  help  students  develop  their  writing  skills  (Jones,  1988).  Peyton 
and  Seyoum  (1989)  pointed  out  that  dialogue  journaling  "held  promise  as  a way  to  promote  the  writing  development  of 
limited  English  proficient  students"  (p.  310).  Research  shows  that  dialogue  journaling  is  an  effective  way  to  help 
improve  ESL  student  writing  (Blanton,  1987;  Gross,  1990;  Gunkel,  1991;  Gutstein,  1983;  Reyes,  1991;  Peyton, 

Staton,  Richardson  & Wolfram,  1990;  Rinvolucri,  1983;  Spack  & Sadow,  1983;  Staton,  1988;  Staton,  Shuy,  & 
Kreeft,  1982;  Steer,  1988;  Urzua,  1987;  Venditti  & Bahruth,  1986). 

To  date,  the  communication  medium  used  for  dialogue  journaling  has  been  paper  and  pencil.  The  advent  of 
widely  available  computers  and  telecommunications  has  created  opportunities  to  take  dialogue  journal  writing  one  step 
further  through  use  of  e-mail. 

E-mail  is  a unique  communication  medium.  "It  differs  from  any  other  communication  in  time,  space,  speed, 
ease  of  use,  fun,  audience,  and  opportunity  for  feedback  "(Kiesler,  1984;  p.  1 127).  As  a written  communication  medium, 
e-mail  not  only  allows  the  writer  to  interact  with  the  text  via  die  computer,  but  also  makes  it  possible  and  easy  for 
writers  to  interact  with  each  other  via  the  computer.  E-mail  enhances  the  communication  between  writers  by  transferring 
their  writings  to  each  other  almost  spontaneously.  Equipped  with  word  processing  capabilities,  e-mail  smoothes  the 
writing  process  of  the  writer.  Thus,  the  computer  becomes  "a  communication  channel  as  well  as  a writing  tool"  (Daiute, 
1985,  p.  xiv). 

E-mail  has  been  reported  as  an  effective  communication  medium  in  a variety  of  educational  settings.  Surveys 
conducted  across  campuses  in  the  United  States  reveal  that  teachers  and  students  use  e-mail  as  a communication  tool  to 
fulfill  their  personal  needs.  E-mail  is  used  for  exchanging  information,  discussing  opinions,  and  socializing  with  people 
(Grabowski,  1990;  Rice  & Case,  1983).  A survey  by  Schaeferrneyer  and  Sewell  (1988,  p.  1 19)  identified  the  advantages 
of  e-mail  as  “speed,  “ “convenience,  “ “time-saving,  “ and  “asynchronicity.”  Grabowski  (1990,  p.  280)  identified  the 
reasons  for  using  e-mail  as  : “no  time/place  limits,”  “easy  access,”  “knowledge  of  computers,”  “ease  of  use,” 
“convenience,”  and  to  “try  new  things.”  These  surveys  indicate  that  e-mail  is  perceived  by  teachers  and  students  as  an 
accepted  and  valuable  tool  for  personal  communication. 

E-mail  communication  is  sometimes  added  to  the  regular  classroom  in  the  hope  of  increasing  the  interaction 
between  teachers  and  students,  and  thus  improving  the  quality  of  the  learning  environment.  Studies  report  that  students 
demonstrate  an  overwhelming  acceptance  of  e-mail  as  a supplement  to  traditional  modes  of  instructor-student  interaction 
(Downing,  Schooley,  Matz,  Nelson  & Martinez,  1988;  D’Souza,  1991;  Hartman  et  al,  1991;  Kinkead,  1987).  Students 
described  e-mail  as  "quick,  convenient,  and  fun"  (Kinkead,  1987,  p.  338).  The  teachers  interacted  more  frequently  with 
their  students  using  e-mail  than  they  did  in  face-to-face  or  paper  communication  (Hartman  et  al,  1991).  Students  asked 
more  challenging  and  more  thoughtful  questions,  and  the  instructor’s  responses  to  these  questions  were  often  of  higher 
quality  than  those  they  provided  in  face-to-face  interaction  (Downing,  Schooley,  Matz,  Nelson  & Matinez,  1988).  The 
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quality  of  instruction  and  student  test  scores  improved  because  of  the  increased  interaction  between  instructors  and 
students  (D'Souza,  1991). 

. E-mail  expands  the  communication  circles  of  the  learner  beyond  the  boundaries  of  the  classroom  and  the 
campus.  The  learner  is  provided  access  to  global  communication  and  empowered  in  the  learning  process.  Jenkinson 
(1992)  reported  a study  on  e-mail  communication  between  a university  professor  and  20  students  (fourth  through  seventh 
graders).  The  students  sent  their  stories  to  the  professor.  The  professor  critiqued  their  stories  via  a home  computer  and 
sent  suggestions  back  to  the  students.  Most  of  the  students  revised  their  writings  following  the  professor  s suggestions. 

E-mail  links  students  together  from  different  cultures.  The  Intercultural  Learning  Network  is  an  international 
computer  network  that  was  developed  to  provide  students  from  different  cultures  the  opportunity  to  work  together  on 
joint  educational  activities  (Cohen,  1986).  Students  in  Mexico,  Japan,  Israel,  and  the  United  States  participated  in  the 
project.  The  students  in  Japan  used  English,  the  second  language  they  were  studying,  to  complete  their  projects. 

Students  showed  a great  motivation  to  use  English  to  communicate  through  network  activities.  Test  scores  demonstrate 
that  the  students  in  Japan  gained  English  proficiency  through  these  network  activities.  The  students  agreed  that  the 
language  they  learned  in  networking  activities  was  a living  language. 

In  the  study  conducted  by  Soh  and  Soon  (1991),  two  groups  of  teenage  students  from  two  cultures  (Singapore 
and  Canada)  communicated  via  e-mail.  Researchers  found  that  the  students  learned  to  write  clear  and  effective  prose  to 
communicate  their  ideas  and  concepts.  In  addition,  the  students  learned  that  the  computer  can  be  an  effective 
communicative  tool  as  well  as  a learning  tool. 

Although  a search  of  literature  failed  to  locate  any  research  on  using  e-mail  as  a writing  medium  in  dialogue 
journaling,  the  literature  discussed  above  establishes  the  educational  value  of  using  e-mail  and  dialogue  journaling  on 
paper  as  educational  tools.  The  present  study  involved  designing  an  electronic-based  dialogue  journaling  environment  to 
explore  whether  the  two  tools  can  be  combined  to  create  an  effective  learning  environment  in  the  ESL  writing  classroom. 

The  Study 

This  case  study  involved  designing  an  electronic-based  environment  to  explore  the  effectiveness  of  e-mai)  as  a writing 
tool  for  dialogue  journaling.  The  issues  investigated  were:  What  were  the  students’  attitude  toward  dialogue  journal 
writing  via  e-mail?  What  was  the  instructor’s  perception  regarding  e-mail  dialogue  journal  writing?  What  problems 
occurred  in  the  process  of  using  e-mail  as  a tool  for  dialogue  journaling?  In  what  ways,  were  e-mail  journals  different 
from  paper  journals? 

Setting  and  Participants 

The  setting  in  the  present  study  was  an  intermediate  level  reading  and  writing  class  in  the  American  English 
Institute  (AEI)  on  the  campus  of  a large  public  university.  The  research  covered  a period  of  nine  weeks. 

There  were  fourteen  students  in  the  class.  Half  of  the  students  were  randomly  assigned  to  the  e-mail  group, 
taking  into  consideration  of  gender  and  nationality..  The  other  half  of  the  students  in  the  class  were  assigned  to  the 
paper-and-pencil  group. 

It  needs  to  be  pointed  out  that  one  student  was  included  into  the  e-mail  group  as  a special  case.  Two  weeks  after 
the  project  started,  one  student  from  the  e-mail  group  dropped.  One  student  (Hiroko)  from  the  paper  group  volunteered  to 
join  the  e-mail  group.  Since  the  nature  of  this  research  is  qualitative,  the  researcher  gave  her  the  permission.  Another 
consideration  was  that  it  might  contribute  insightful  information  to  the  study  to  have  a subject  who  had  experience  using 
both  medium. 

During  the  term,  some  students  dropped  the  class,  some  students  had  long  illness  and  some  students  had  poor 
attendance.  These  students  were  excluded  from  the  sample  in  the  present  study.  The  final  student  samples  are  shown  in 
table  1 and  table  2. 
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TABLE  1.  Student  Sample  (E-mail  Group) 


Student 

Nationality 

Gender 

Final  grade  for  the 
term 

Ying 

Taiwan 

Female 

88.8 

Lin 

Taiwan 

Male 

90.5 

Toshi 

Japan 

Male 

67.8 

Tamiko 

Japan 

Female 

85.3 

Hiroko 

Japan 

Female 

91.6 

Andi 

Indonesia 

Male' 

81.0 

TABLE  2. 

Student  Sample  ('Paper  and  Pencil  Groupl 

Student 

Nationality 

Gender 

Final  grade  for  the 
term 

Meimei 

Taiwan 

Female 

84.0 

Shen 

Taiwan 

Female 

92.0 

Atzko 

Japan 

Female 

90.8 

Hiro 

Japan 

Male 

75.3 

Dialogue  Journal  Writing  Procedure 

The  students  in  both  groups  wrote  dialogue  journals  outside  of  class  time.  The  students  in  the  e-mail  group 
sent  their  dialogue  journals  to  the  instructor  by  the  use  of  t-maii  every  day.  They  used  the  computer  lab  in  AEI  program 
on  the  campus.  The  students  in  the  paper-and-pencil  group  wrote  their  dialogue  journals  on  pieces  of  paper  and  handed 
them  in  to  the  instructor  every  morning  before  the  class  started. 

The  instructor  used  e-mail  to  write  to  the  students  in  the  computer  group  and  an  electronic  typewriter  to  write  to  the 
students  in  the  paper  group. 

Data  Collection 

A triangulation  strategy  was  used  for  data  collection  in  the  present  study.  Data  collection  methods  in  the  present 
study  included:  (a)  classroom  and  lab  observation;  (b)  interviews;  and  (c)  the  collection  of  the  dialogue  journals  written 
by  the  students  and  the  instructor  in  both  e-mail  group  and  paper  group. 

Classroom  observation 

The  researcher  attended  the  class  three  times  a week  to  do  classroom  observations.  Detailed  field  notes  were 
taken  during  the  observations  on  how  the  class  was  organized,  how  the  instructor  interacted  with  the  students,  and  what 
activities  the  students  did  in  the  class.  Participant  observation  allowed  the  researcher  to  view  the  research  in  a broader 
context  and  provided  the  researcher  a better  understanding  of  how  other  factors  might  affect  the  results  of  the  research. 

Interviews 

The  researcher  did  informal  interviews  with  the  students  and  the  instructor  before  class  started  or  during  class 
breaks.  A semi-structured  interview  was  conducted  at  the  end  of  the  project  and  follow-up  telephone  interviews  were 
conducted  four  months  later  after  the  project  was  over. 

Collection  of  the  dialogue  journals 

An  aliases  file  was  built  into  the  e-mail  account  of  each  student  and  the  instructor  in  order  to  collect  the  dialogue 
journals  written  by  the  students  and  the  instructor  via  e-mail.  The  aliases  were  designed  so  that  the  dialogue  journal 
entries  written  by  the  students  and  the  instructor  via  e-mail  were  automatically  sent  to  the  researcher  as  well  as  to  their 
specified  receiver. 

The  paper  journals  were  collected  weekly.  Each  Friday  the  students  in  the  paper  group  were  required  to  bring  all 
the  journals  they  wrote  during  the  week  and  hand  them  in  to  the  instructor.  The  instructor  made  copies  of  the  paper 
journals  and  gave  the  copies  to  the  researcher. 
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Dflta  Analysis 

Data  analysis  was  based  on  interviews,  field  notes,  and  the  written  dialogue  journals  (e-mail  and  paper  and 
pencil)  produced  by  the  students  and  the  instructor.  Data  was  categorized  and  organized  to  describe  the  subject’s  attitude 
towards  using  e-mail  as  a medium  for  writing  the  dialogue  journal  and  the  problems  that  the  participants  encountered  in 
the  process  of  using  e-mail. 

To  compare  e-mail  journals  and  paper  journals,  Shuy’s  model  on  sentence  level  language  function  serve  as  the 
guideline  for  coding  the  dialogue  journals.  Language  functions  are  the  ways  in  which  people  use  language  to  get  things 
done:  "They  may  want  to  tell  something,  to  answer  a question,  complain,  apologize,  promise  something,  or  any  number 
of  other  things.  These  are  all  referred  to  as  language  functions"  (Shuy,  1988,  p.  107).  Shuy  (1982,  1988)  developed  his 
model  in  analysis  of  the  dialogue  journals  written  by  a teacher  and  10  students  over  a period  of  two  weeks  in  the  fall  and 
in  the  spring.  "This  analysis  of  language  functions  is  based  on  the  strong  belief  that  language  functions  are  a more 
effective  measure  of  writing  abilities  than  any  existing  measure  of  language  forms"  (Shuy,  1988;  p.142).  Ten  frequently- 
used  language  functions  were  identified  in  the  dialogue  journals  in  the  present  study:  (a)  reporting  opinion,  (b)  reporting 
personal  facts,  (c)  reporting  general  facts,  (d)  responding  to  questions,  (e)  predicting,  (f)  complaining,  (g)  giving 
directives,  (h)  apologizing,  (i)  thanking,  (j)  offering,  (k)  promising,  and  (1)  requesting. 

The  reliability  of  data  coding  was  checked  by  hiring  the  other  coder  to  code  the  data.  One  hundred-twenty 
sentences  (2%)  were  randomly  selected.  The  methodology  used  for  landom  selection  was  stratified  random  sampling. 

First,  the  total  number  of  the  sentences  was  divided  by  the  number  of  the  sample  to  be  selected  (5812/120=48).  Second, 
the  sentences  were  grouped  according  to  the  categories  of  language  functions.  Different  categories  of  language  functions 
were  entered  into  different  computer  files.  Third,  the  sample  was  selected  consisting  of  each  48th  sentence  in  each 
category  of  language  functions. 

After  the  training,  the  coder  coded  the  data  independently  according  to  the  coding  rules.  The  results  of  his  coding 
showed  that  out  of  120  sentences,  the  coder  correctly  coded  106  sentences  The  reliability  of  the  data  coding  in  the 
present  study  was  88.3%  (106/120=.883). 


Findings  and  Discussions 

Attitude  towards  using  e-mail  in  dialogue  journaling 

Five  out  of  six  students  expressed  a positive  attitude  towards  using  e-mail  in  dialogue  journaling.  They  were 
Lin,  Andi,  Hiroko,  Toshi  and  Ying.  They  preferred  using  e-mail  to  using  paper  and  pencil.  Ying  enjoyed  using  e-mail, 
but  she  mentioned  she  would  not  mind  using  paper  and  pencil  as  well  in  writing  dialogue  journals.  These  students 
commented  that  e-mail  is  fast,  fun  and  convenient.  Tamiko  held  an  attitude  of  total  rejection  towards  using  e-mail. 

Both  Lin  and  Andi  had  computer  background  and  typing  skills.  They  expressed  positive  attitudes  towards  the 
use  of  e-mail.  Based  on  observations  in  the  computer  lab  during  e-mail  training,  both  students  handled  the  dialogue 
journal  typing  task  with  ease.  Each  of  them  spent  about  10  minutes  a day  in  writing  their  dialogue  journals.  The  10 
minutes  figure  is  in  sharp  contrast  with  the  average  of  30  minutes  spent  by  the  other  e-mail  users  and  the  paper-and- 
pencil  students.  These  two  students  enjoyed  using  e-mail  because  they  were  familiar  with  the  computer  technology  and 
they  did  not  have  difficulties  in  typing.  They  felt  it  was  easy  and  convenient  for  them  to  write  via  e-mail. 

When  I use  e-mail,  I have  enough  time  and  it  is  easier  to  edit.  I 

feel  it  is  better  for  me.  I feel  it  is  very  troublesome  to  write  by  hand.  When  I write  by  hand,  I have  to 
erase  by  using  the  eraser.  After  that,  I have  to  try  to  make  the  paper  look  clean.  I really  feel  it  is  very 
troublesome  to  do  so.  My  handwriting  is  very  poor.  It  was  also  faster  for  me  to  write  by  the  computer 
than  to  write  by  hand.  (Lin) 


Researcher: 
Andi : 
Researcher: 
Andi : 


What  did  you  like  most  about  e-mail  writing  ? 
Typing . 

Typing ? 

Yes. 
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Researcher:  You  like  typing  better  than  handwriting? 

You'd  rather  type  than  to  write  by  hand? 

Andi:  Yes. 

Among  the  students  who  did  not  have  computer  background  and  typing  skills,  Toshi,  Hiroko  and  Ying 
expressed  positive  attitudes  towards  e-mail.  Toshi,  Hiroko  and  Ying  were  very  enthusiastic  about  learning  the  computer 
technology  and  typing  skills.  During  the  interviews,  these  three  students  identified  that  by  doing  e-mail  dialogue 
journals,  they  could  learn  typing  and  computer  technology.  The  following  excerpts  of  their  dialogue  journals  indicated 
that  they  thought  that  it  was  important  for  them  to  learn  the  computer  technology  and  typing  skills  for  their  future 
careers.  Their  attitude  towards  e-mail  was  affected  by  their  desire  to  learn  the  computer  technology  and  typing  skills 
which  they  considered  relevant  to  their  career  goals. 

(Ying) 

It  is  a nice  thing  that  I can  talk  to  you  by  using  my  fingers.  This  is  a very  special  experience  to  me.  If  I can  be 
a regular  student,  I might  have  my  own  e-mail  account.  I hope  I still  can  talk  with  you. 

(Hiroko) 

Hi!  This  is  Hiroko.  This  is  the  first  time  to  send  you  my  letter.  Since  I only  learned  how  to  use  computer 
yesterday,  I ought  to  be  at  a loss.  However,  I think  its  very  important  to  learn  to  use  computers  which  are 
indispensable  to  future.  And  that,  secretary  whom  I want  to  be  also  need  the  skills  of  using  computer.  Now  it 
is  the  time  to  learn  it.  I will  do  my  best!!  See  you. 

(Toshi) 

I am  sorry.  I forgot  to  reply  your  message.  Here  are  my  answers.  I like  to  use  computer.  Therefore,  I used 
computer  every  day.  Also,  I like  to  write  something.  So,  I hope  that  I want  to  use  computer  next  term.  I was 
looking  for  use  computer  and  to  communication  by  write  to  American  people.  If  you  can  do  this  program  next 
term,  please  tell  me. 

E-mail  participants  considered  it  an  advantage  that  they  did  not  need  to  coordinate  a time  to  communicate  with 
each  other,  they  did  not  need  to  deal  with  paper  work  and  they  could  concentrate  on  their  writing  in  an  e-mail 
environment. 

I saw  another  group  turned  in  their  journals  early  in  the  morning  in  class  every  day.  I feel  it  is  very 
troublesome.  I don't  like  to  do  that.  It  turns  the  dialogue  journal  writing  into  formal  homework.  It  is  very 
convenient  for  me  to  use  the  computer.  All  I need  to  do  is  to  turn  on  the  computer.  (Lin) 

Andi:  ...  because  when  / was  writing  in  my  apartment , / leave  my  writing , and  I'll  go  anywhere. 

After  that,  Vll  continue  again.  Then  / have  something  to  do,  and  then  back  to  write. 

Researcher:  So  you  think  there  are  lots  of  distractions  at  home?  But  with  the  computer ; you  can 

concentrate,  just  ten  minutes? 

Andi:  Yeah,  just  ten  minutes. 

Researcher:  OK.  What  inconvenience  does  e-mail  bring  to  your  writing? 

Hiroko:  Eh.... 

Researcher:  Compared  with  writing  by  paper  and  pencil? 

Hiroko:  Oh.  / think  writing  in  paper  is  more  inconvenient.  / think  e-mail  is  more  convenient. 
Researcher:  Can  you  tell  me  why? 
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Hiroko:  Because,  l do  not  need  to  prepare  drafts.  Paper  writing,  I have  to  write  the  day  before.  In  the 
morning,  l have  to  submit  to  the  instructor.  But  e-mail  is. ..can  send  thinking  fast..  .1  think  e-mail  is 
more  convenient. 

Tamiko  held  a negative  attitude  towards  using  e-mail  as  a writing  medium  in  dialogue  journaling.  She  thought 
e-mail  is  "so  so."  She  did  not  like  to  use  e-mail  to  write  her  dialogue  journal.  She  felt  the  computer  letters  were 
impersonal.  In  addition,  she  even  did  not  want  to  consider  using  e-mail  as  a communication  tool  in  the  future. 

Another  frustration  that  Tamiko  felt  about  using  e-mail  was  that  she  could  not  make  corrections,  even  when  she 
found  the  mistakes  while  she  was  writing  via  e-mail: 

Tamiko's  view  about  errors  affected  her  attitude  towards  the  dialogue  journal.  She  had  considerable  anxiety 
about  the  errors  she  made  in  her  writing.  From  the  very  beginning,  she  requested  the  instructor  to  correct  her  writing 
errors  in  her  dialogue  journals: 

(Tamiko) 

Hello!  Karen.  How  are  you.  I am  very  fine  now,  because  I am  done  all  my  class  and  test.  However  I already 
found  some  mistakes.  By  the  way,  I have  question.  I don't  know  what  should  I do  for  my  free-writing  [the 
dialogue  journal],  I think  I always  have  many  mistake  in  my  writing.  However  I don't  know  what  is  my 
mistake.  So,  could  you  tell  me  about  that?  See  you  next  Monday.  Have  a nice  weekend!  Bye! 

Tamico’s  attitude  towards  e-mail  was  affected  by  her  error  phobia,  her  attitude  towards  dialogue  journaling  and 

this  was  compounded  by  her  lack  of  typing  and  computer  skills. 

If  viewed  from  a broader  context,  Tamico’s  attitude  was  affected  by  a powerful  reality:  the  TOEFL  test,  a test 
required  for  all  foreign  students  who  want  to  be  admitted  to  universities  and  colleges  in  North  America.  The  classroom 
observation  indicated  that  the  classroom  instruction  was  attuned  to  the  requirements  of  the  TOEFL  test,  that  focuses  on 
grammar  and  vocabulary.  The  approach  of  dialogue  journaling  in  language  learning,  however,  is  in  conflict  with 
preparing  to  pass  a TOEFL  test.  In  informal  interviews  and  in  their  journal  entries,  the  students  openly  expressed  their 
worries  about  spending  too  much  time  writing  dialogue  journals  which  was  irrelevant  to  preparation  for  the  TOEFL  test. 
Comparatively  speaking,  the  students  in  the  e-mail  group  were  facing  a greater  challenge  than  the  students  in  paper 
group.  The  students  in  e-mail  group  had  to  handle  an  entirely  new  writing  environment. 

The  instructor  enjoyed  using  e-mail  to  conduct  dialogue  journaling.  When  asked  to  compare  the  two  mediums, 
the  instructor  thought  she  had  better  communication  with  the  students  in  the  computer  group  than  with  the  students  in 
the  paper  group.  In  fact,  she  spent  more  time  writing  to  the  students  in  the  computer  group. 


Karen: 


Researcher: 

Karen: 

Researcher: 

Karen: 


I think  I spend  more  time  on  the  computer  group.  And  it  was  direct  communication. 
They  wrote  to  me  and  I wrote  back,  the  same  topic,  back  and  forth,  back  and  forth.  Well, 
with  the  paper  group,  we  were  discussing  one  topic  on  Monday  and  what  I received  on 
Tuesday  morning  was  a different  topic,  so  sometimes  I did  not  have  too  much  to  say,  and 
then  they  go  back  on  Monday's  topic,  but  I would  forget  what  I said  on  Monday.  So  it 
wasn’t  a good  flow  of  communication  with  the  paper  group. 

So  which  group  do  you  spend  more  time  writing  to? 


With  the  e-mail  group,  I think. 

Oh.  You  spend  more  time  because  you  have  something  to  say  to  them? 

Yeah.  Because  I would  ask  them  questions  and  they  responded  to  the  questions,  back  and 
forth,  back  and  forth.  But  with  the  other  group,  I asked  them  questions  on  Monday,  and  I 
wouldn't  get  response  till  Wednesday.  So  it  just  didn't  seem  as  communicative  as  e-mail. 

Another  reason  why  the  instructor  liked  to  use  the  computer  for  dialogue  journaling  was  that  she  could  have  all 
the  students'  writings  saved  in  the  computer,  and  it  was  convenient  for  her  to  refer  to  what  the  student  had  previously 
written. 


Researcher: 

Karen: 

Researcher: 

Karen: 

Researcher: 

Karen: 


How  would  you  compare  the  two  different  mediums?  You  are  using  the  computer  and 
you  are  using  typewriter. 

How  would  I compare  them? 

Yeah.  The  computer  medium  and  the  paper  medium. 

OK.  I have  not  thought  about  this  before.  I prefer  the  computer  for  almost  everything 
that  I do,  but  I don't  know  exactly  why.  If  I say  it  saves  time,  then  it  is  true.  But  I don't 
know  if  it  was  expected  to  save  time. 

Did  you  feel  it  was  troublesome  to  exchange  folders  with  the  paper  group  every  day? 

But  I have  been  doing  this  so  long.  You  know,  it  would  be  nice  not  to  have  to  distribute 
folders  every  day,  collect  and  return.  That  sounds  good.  I know  one  thing  that  I like 
better  about  the  computer  is  that  I can  keep  all  their  writings  in  the  folders,  that  I can 
look  at  them  at  any  time,  whereas  the  paper  group,  once  they  took  back  their  papers,  I 
never  saw  them  again.  So  I can  go  back  in  their  folders,  I can  look  three  or  four  days 
back  if  I wanted  to,  but  I could  not  do  that  with  the  paper  group. 


It  is  obvious  that  each  individual's  attitude  toward  e-mail  was  affected  by  a variety  of  factors,  such  as  personal 
computer  background,  typing  skills,  perception  of  the  writing  process,  attitude  toward  learning  technology,  and  a sense  of 
relevance  of  the  present  project  to  their  imminent  academic  goals  and  future  academic  goals.  These  factors  combined 
together  to  exert  an  influence  on  their  attitude  toward  e-mail  as  a writing  medium  in  dialogue  journaling.  No  single 
factors  alone  determined  their  attitude. 


Problems  encountered  in  using  e-mail 

Typing 

The  students  who  did  not  have  typing  skills  usually  spent  about  thirty  minutes  in  writing  a dialogue  journal 
entry.  Ying  spent  an  even  longer  time  than  that.  She  thought  that  one  of  the  reasons  why  she  spent  so  much  time 
writing  every  day  was  that  she  typed  slowly.  Both  Toshi  and  Hiroko  reported  that  they  spent  time  thinking  about 
keyboarding  while  they  were  writing. 

Limited  Access  to  the  Use  of  Computers 

All  the  students  in  the  study  reported  that  sometimes  a computer  was  not  available  to  use  when  they  went  to  the 
computer  lab  to  write.  During  the  fall  term,  the  computer  lab  in  the  AEI  program  was  very  crowded.  There  was  always 
a line  of  students  waiting  to  use  the  computers.  Tamiko  reported  that  she  twice  missed  writing  because  she  could  not  get 
access  to  a computer.  Other  students  reported  that  they  had  to  spend  a lot  of  time  waiting  in  lines. 

Some  of  the  students  felt  frustrated  with  the  editing  functions  provided  by  the  e-mail  system.  The  e-mail 
system  used  for  the  present  study  did  not  provide  the  types  of  editing  functions  and  other  aids  to  writing  that  one  finds  in 
a modem  word  processor.  The  students  could  not  use  a spelling  checker  to  correct  their  spelling.  The  students  could  not 
use  the  computer  mouse  to  move  the  cursor  to  the  places  where  they  wanted  to  make  changes.  They  had  to  use  the  arrow 
keys  to  move  the  cursor  line  by  line  to  the  place  they  wanted  to  make  changes.  Often  the  appearance  of  the  whole  screen 
display  was  spoiled  when  changes  were  made. 

However,  it  needs  to  be  pointed  out  that  this  editing  problem  is  specific  to  the  e-mail  system  involved  in  the 
present  study.  It  should  not  be  generalized  to  other  e-mail  systems. 

Students’  dissatisfaction  with  the  e-mail  system  sometimes  was  not  caused  by  the  system  itself,  but  by  their 
limited  knowledge  about  the  system.  For  example,  Ying  mentioned  that  she  could  not  refer  to  the  journal  entries  she  had 
previously  made,  because  they  had  been  sent  on  to  the  teacher.  Actually,  it  would  have  been  quite  easy  for  her  to  save  a 
copy  of  each  message  she  sent.  These  e-mail  difficulties  suggest  that  the  initial  training,  although  adequate  to  get 
students  started,  needed  to  be  supplemented  by  additional  training.  The  additional  training  needs  to  review  important 
features  that  students  have  not  learned  to  use,  and  to  present  additional  features  that  they  may  find  useful. 


The  Computer  System  Malfunctioned 

A few  times,  the  participants  reported  that  the  computer  system  malfunctioned.  The  researcher  did  not  work 
with  the  students  in  the  computer  lab  all  the  time,  so  it  was  difficult  to  identify  what  the  specific  problems  were. 
Problems  might  be  caused  either  by  the  computer  network  system  or  by  inappropriate  operations  on  the  part  of  the 
student.  The  computer  problems  caused  frustration  for  the  students,  because  most  of  them  did  not  have  any  computer 
background.  It  should  be  noted  that  the  computer  problems  happened  very  infrequently.  Generally  speaking,  the 
computer  network  system  functioned  well  throughout  the  study. 

Differences  between  e-mail  journals  and  paper  journals 

The  comparison  of  the  language  functions  used  in  e-mail  journals  and  paper  journals  revealed  three  patterns. 

The  participants  in  the  e-mail  group  tended  to  use  formula  function,  requesting  function,  and  tended  to  produce  more 
language  functions  per  writing  session  compared  with  the  participants  in  the  paper  group.  Supported  by  the  other 
findings  of  the  differences  between  e-mail  journals  and  paper  journals,  the  present  study  indicated  that  e-mail  created  a 
different  writing  mode  when  it  was  used  for  dialogue  journaling  in  a classroom  setting. 

Formula  Functions 

Formula  Functions  refer  to  openings  and  closings  in  dialogue  journal  entries.  The  students  in  the  e-mail  group 
tended  to  use  formula  functions  while  the  students  in  the  paper  group  did  not  (see  table  3).  The  students  in  the  e-mail 
group  often  greeted  the  instructor  at  the  beginning  of  their  writings:  "Hi!  How  are  you  today?",  "Good  morning, 
Karen.",  "Hello,  Karen,  this  is  Lin.",  "Hi!  This  is  Toshi.  How  are  you  doing?".  They  used  closings  at  the  end  of  their 
writings:  "See  you  tomorrow,"  "See  you  in  class,"  "Bye-bye,"  "Have  a good  day.”  None  of  the  students  in  the  paper 
group  used  any  of  these  formula  functions. 

The  instructor  used  more  formula  functions  in  writing  to  the  students  in  the  e-mail  group  than  did  she  to  the 
students  in  the  paper  group  (see  table  4). 

TABLE  3.  Ranked  Comparison  of  Formula  Functions  Used  by  the  Students 

in  the  Two  Groups 


Student 

Opening 

N 

Closing 

N 

Total 

N 

Formulas  Per 
Writing  Session 
N 

E-mail  Group 
Lin  (36) 

60 

28 

88 

2.44 

Tamiko  (38) 

53 

55 

88 

2.32 

Ying  (47) 

11 

45 

56 

1.19 

Toshi  (42) 

14 

22 

36 

0.86 

Hiroko  (24) 

1 

9 

10 

0.42 

Andi  (29) 

0 

0 

0 

0.00 

M= 

23.17 

26.5 

46.33 

1.21 

Paper  and  Pencil 

Group 

Atzko(37) 

0 

0 

0 

.00 

Shen  (37) 

0 

0 

0 

.00 

Meimei  (27) 

0 

0 

0 

.00 

Hiro  (20) 

0 

0 

0 

.00 

M= 

0 

0 

0 

.00 

Note:  Numbers  in  parentheses  are  the  total  numbers  of  writings  each  student  did  in  the  program. 


TABLE  4.  Ranked  Comparison  of  the  Formula  Function  ’’Closing"  Used  by  the  Instructor  Per  Writing  Session  in 

Response  to  the  Students  in  the  Two  Groups 


Instructor  to  Student 

Closing  (Total 
Numbers) 

N 

Numbers  Used  Per 
Writing  Session 
N 

E-mail  Group 
Lin  (36) 

15 

.42 

Ying  (47) 

14 

.30 

Toshi  (42) 

11 

.26 

Tamiko  (38) 

11 

.29 

Hiroko  (24) 

5 

.21 

Andi  (29) 

5 

.17 

Paper  and  Pencil  Group 
Atzko(37) 

4 

.11 

Shen  (37) 

4 

.11 

Meimei  (27) 

2 

.07 

Hiro  (20) 

0 

.00 

The  finding  that  the  students  in  the  paper  group  did  not  use  formula  functions  agrees  with  the  findings  of 
previous  studies  conducted  on  paper  journals.  In  both  Shuy’s  studies  (1982,  1988)  and  Gutstein’s  study  (1983),  the 
students  used  formula  functions  infrequently  in  their  paper-and-pencil  dialogue  journals. 

Due  to  its  perceived  speed,  e-mail  writing  in  fact  closely  parallels  a phone  conversation  or  face-to-face 
conversation  (Baron,  1984;  Bolton,  1991;  Dubin,  1991;  Murray,  1991).  In  a phone  conversation  or  face-to-face 
conversation,  people  frequently  use  formula  functions. 

Though  the  participants  in  the  e-mail  group  tended  to  use  formula  functions  in  their  writings,  they  did  not 
consider  it  a rule  for  them  to  follow.  They  were  flexible  in  using  formula  functions.  Sometimes  they  used  both 
openings  and  closings.  Sometimes,  they  used  only  openings  or  only  closings.  One  student  never  used  any  openings  and 
closings  in  his  messages. 

Requesting  Functions 

Another  obvious  difference  is  that  the  students  in  the  computer  group  asked  more  questions  than  did  the  students 
in  the  paper  group  (see  table  5). 


TABLE  5.  Ranked  Comparison  of  Questions  Asked  by  the  Students  in  the  Two  Groups 


Student 

Total  Questions 
N 

Questions  Per  Writing 
Session 

N 

E-mail  Group 
Ying  (47) 

57 

1.21 

Hiroko  (24) 

27 

1.13 

Tamiko  (38) 

21 

0.55 

Andi  (29) 

15 

0.52 

Toshi  (42) 

17 

0.40 

Lin  (36) 

12 

0.33 

M= 

24.83 

0.69 

Paper  and  Pencil  Group 
Atzko(37) 

12 

0.32 

Shen  (37) 

9 

0.24 

Meimei  (27) 

3 

0.11 

Hiro  (20) 

0 

0.00 

M= 

6 

0.17 

This  finding  is  consistent  with  the  findings  of  previous  studies  conducted  on  using  e-mail  as  a communication 
medium  in  a variety  of  settings.  The  previous  studies  indicate  that  e-mail  users  tend  to  ask  questions  and  seek 
information  in  e-mail  writings.  Rice  and  Case  (1983)  did  a study  on  the  patterns  of  e-mail  communication  among  e-mail 
users  on  the  campus  of  a university.  They  found  that  the  highest  uses  of  e-mail  were  to  exchange  information  ( 100%), 
ask  questions  (95%),  discuss  opinions  and  keep  in  contact  (84%).  Schaefermeyer  and  Sewell  (1988)  conducted  a survey 
on  how  e-mail  was  used  among  e-mail  users  on  BITNET  (one  of  the  largest  academic  networks  in  the  world).  The  results 
of  their  survey  showed  that  the  major  use  of  e-mail  was  to  seek  informatic  n.  Grabowski  et  al.  (1990)  conducted  another 
study  on  the  use  of  e-mail  among  graduate  students  in  a university.  They  tiund  that  exchanging  academic  information 
(100%)  and  discussing  ideas  (63%)  comprised  the  highest  uses  of  e-mail  in  the  population  of  their  study.  Sherblom 
(1988)  did  a content  analysis  of  the  e-mail  messages  received  by  i middle-level  manager  in  a large  organization  over  the 
period  of  several  months.  The  findings  of  the  study  showed  that  requesting  information  and  providing  information 
comprised  the  largest  part  of  the  content  of  all  the  e-mail  messages  received  by  the  manager. 

The  fact  that  e-mail  participant  tended  to  ask  more  questions  could  be  due  to  their  perceptions  of  the  medium. 
E-mail  is  fast.  When  a person  asks  questions,  the  usual  desire  is  to  receive  responses  as  quickly  as  possible.  The  e-mail 
user  may  have  a perception  that  a message  is  being  sent  rapidly  and  is  immediately  received  by  the  addressee.  This 
perception  of  speed  fits  the  student's  need  to  get  immediate  responses  to  their  questions.  In  actuality,  the  instructor  only 
responded  to  the  student's  writing  once  a day.  The  students  usually  got  the  responses  from  the  instructor  the  next  day.  It 
is  the  students'  perception  of  the  medium  that  facilitated  question  asking. 

The  perception  of  speed  may  be  linked  to  the  feeling  that  once  an  e-mail  message  is  sent,  a communication  task 
has  been  completed.  This  perception  contrasts  with  writing  on  paper  where  it  is  possible  to  continue  to  make  additions 
until  the  paper  is  physically  turned  in  to  the  teacher.  This  delay  might  account  for  the  fact  why  the  paper  group  students 
asked  so  few  questions. 

Overall  Language  Functions 

Comparison  of  the  language  functions  used  in  the  e-mail  dialogue  journals  and  the  paper  dialogue  journals 
indicated  that  the  students  in  the  e-mail  group  either  produced  as  many  language  functions  or  they  produced  more 
language  functions  per  writing  session  than  did  the  students  in  the  paper  group  (see  table  6).  The  instructor  produced  a 
greater  number  of  language  functions  per  writing  session  in  the  e-mail  group  than  in  writing  to  the  students  in  the  paper- 
and-pencil  group  (see  table  7). 

The  conversational  and  informal  style  of  e-mail  dialogue  journaling  might  have  encouraged  the  students  and  the 
instructor  to  write  more.  E-mail  dialogue  journaling  is  closer  to  oral  communication  than  paper  dialogue  journaling  is. 
E-mail  participants  do  not  need  to  pay  as  much  attention  to  the  rules  that  are  required  in  formal  writings.  In  Kinkcad's 
study,  she  observed  the  conversational  nature  of  e-mail  writings:  'These  writers  often  refer  to  E-mail  as  holding  a 


conversation  since  its  informality  almost  demands  that  they  write  the  way  they  talk,  breaking  down  the  rigid  rules  of 
more  formal  communication**  (Kinkead,  1987,  p.  341). 

The  informal  and  relaxing  environment  facilitated  communication  between  the  instructor  and  the  students.  The 
students  lost  the  constraints  and  inhibitions  the  intended  audience  provided.  They  no  longer  feel  bound  by  formal 
instructor-student  relationship.  As  one  student  mentioned  in  her  e-mail  journal: 

It  is  a nice  thing  that  I can  talk  to  you  by  using  my  fingers. 

I like  to  use  this  way  talk  with  you.  This  is  a very  special  experience  to  me.  You  know?  In  my  country, 

students  have  to  call  teacher  "Teacher",  sounds  very  serious.  (Ying) 

E-mail  made  the  process  of  communication  between  the  students  and  the  instructor  easy  and  convenient.  Some 
students  mentioned  that  e-mail  is  fast  and  convenient.  The  students  and  the  instructor  did  not  need  to  coordinate  a time  to 
communicate  with  each  other.  Both  the  students  and  the  instructor  considered  it  an  advantage  that  they  did  not  need  to 
deal  with  paper  work  when  they  wrote  via  e-mail.  This  added  convenience  may  have  encouraged  them  to  write  more. 

The  computer  itself  probably  provided  a motivation  for  the  students  and  the  instructor  to  write  more.  The 
instructor  and  most  of  the  students  indicated  that  they  enjoyed  working  with  the  computer.  Some  of  the  students  felt  it 
was  fun  to  write  using  the  computer.  Some  of  the  students  considered  it  important  for  them  to  learn  to  use  the  computer 
technology.  They  were  willing  to  spend  more  time  writing  with  the  computer  because  they  felt  the  task  of  learning  the 
technology  was  relevant  to  their  future  careers. 

TABLE  6.  Ranked  Comparison  of  the  Means  of  Language  Functions  Use  by  the  Students  in  the  Two  Groups  Per 

Writing  Session 


Frequency  of  Language  Function  Use  for  Each  Student 

Total  of  Language  Function 
Use 

Mean  of  Language  Function 
Use  per  Writing  Session 

E-mail  Group 
Ying  (47) 

768 

16.34 

Lin  (36) 

573 

14.74 

Hiroko  (24) 

531 

14 

Toshi  (42) 

515 

13.63 

Tamiko  (38) 

336 

13.55 

Andi  (29) 

253 

8.72 

Paper  and  Pencil 

Group 

Shen  (37) 

379 

10.24 

MeiMei  (27) 

333 

12.33 

Atzko  (37) 

315 

8.51 

Hiro  (20) 

228 

11.44 

777 


TABLE  7.  Ranked  Comparison  of  the  Means  of  the  Language  Functions  Used  by  the  Instructor  to  the  Students  in  Both 

Groups  Per  Writing  Session 


Frequency  of  Language  Function  Use  for  Each  Student 


Total  of  Language  Function 
Use 


E-mail  Group 
Ying  (47) 

Lin  (36) 
Hiroko  (24) 
Toshi  (42) 
Tamiko  (38) 
Andi  (29) 

311 

191 

135 

191 

193 

148 

Paper  and  Pencil 

Group 

Shen  (37) 

142 

MeiMei  (27) 

93 

Atzko  (37) 

118 

Hiro  (20) 

59 

Mean  of  Language  Function 
Use  per  Writing  Session 

6.62 

5.31 

5.63 

4.55 

5.08 

5.10 


3.84 

3.44 

3.19 

2.95 


Other  Findings 

The  present  study  also  indicates  that  e-mail  creates  a different  writing  mode.  Conversational  cues  were  found  in 
e-mail  messages.  In  e-mail  dialogue  journals,  the  students  and  the  instructor  used  conversational  cues  such  as  "Wow", 
"OOPS",  and  "Yuck." 


(Ying) 

Wow!  I am  very  glad  since  you  told  me  such  clear  and  interesting  answers. 


(Hiroko) 

I will  go  to  the  football  game  on  Saturday  and  also  go  shopping  on  Sunday.  Besides,  my  friend  when  I was 
high  school  student  will  come  to  Eugene.  So  I somehow  can  break  off  my  frustration  during  this  weekend. 
OOPS!!  I have  two  TEST  next  week! 

(Instructor) 

Lice!  Yuck!  Yuck!  Kids  in  public  schools  often  get  this  and  it  is  very,  very  difficult  to  get  rid  of  it. 


The  students  in  the  e-mail  group  in  the  present  study  wrote  more  informally  and  casually  than  did  the  students  in 
the  paper-and-pencil  group.  They  wrote  as  if  they  were  holding  a conversation.  “Since  e-mail  often  flies  between  parties 
at  a rate  approaching  a conversation,  and  since  most  people  are  more  comfortable  being  friendly  than  combative,  many 
people  tend  to  drift  into  informality  in  their  electronic  messaging”  (Krol,  1992,  p.  93).  Comparing  the  difference 
between  e-mail  writing  and  paper  writing,  Hawisher  and  Moran  (1993,  p.630)  pointed  out:  “E-mail  seems  now  to  employ 
a language  that  is  somewhere  on  the  continuum  between  spoken  and  written  language....  But  often,  e-mail  is  composed 
on-line,  and  rapidly.  Typically  it  is  not  subjected  to  the  reflective  scrutiny  we  usually  give  to  the  language  we  inscribe 
on  paper.” 

The  sentence  structures  in  the  e-mail  dialogue  journals  were  looser  than  those  in  the  paper  writings.  Indenting, 
paragraphs,  and  punctuation  were  used  very  casually.  The  students  in  the  paper  group  arranged  their  sentences  more 
carefully,  as  if  they  were  doing  formal  writing  assignments. 

The  writing  process  can  also  explain  the  differences  between  e-mail  journals  and  paper  journals.  Interviews 
(both  in'  - mal  and  semi-structurcd)  revealed  that  paper  group  studi  its  used  dictionaries  and  grammar  books  quite 
frequently  while  they  were  writing  dialogue  journals.  Sometimes,  they  wrote  drafts.  E-mail  writing  requires  a high 
degree  of  the  engagement  of  mind  and  hands.  An  e-mail  writing  environment  does  not  encourage  students  to  use 
dictionaries,  grammar  books  or  write  a draft  as  it  did  with  the  students  in  the  paper  group.  One  e-mail  participant  used  a 


™,,a2 


o 


dictionary  a couple  of  times  at  the  beginning  and  gave  it  up  because  it  slowed  him  down  in  his  composing.  Hiroko  was 
an  interesting  case.  She  used  to  write  by  using  paper  and  pencil  for  two  weeks  and  then  she  volunteered  to  joined  e-mail 
group.  When  she  wrote  by  the  use  of  paper  and  pencil,  she  always  wrote  drafts.  She  spent  about  30  to  60  minutes  to 
write  her  paper  journals.  When  she  wrote  by  the  use  of  e-mail,  she  did  not  prepare  drafts  and  she  spent  less  time.  When 
asked  whether  she  wrote  the  same  way  by  the  use  of  the  two  different  mediums,  Hiroko  answered  that  she  did  not  think 
so. 

Researcher:  Yeah.  Because  you  have  this  experience,  at  the  beginning,  you  wrote  by  using  paper 

and  pencil,  and  then  you  moved  into  the  computer  group.  So  l want  to  know  what  you  think.  Do  you 
think  you  wrote  the  same  way  by  using  the  two  mediums? 

Hiroko:  No. 

Researcher:  Can  you  tell  me  the  difference? 

Hiroko : Writing  using  the  paper  is  to  have  a time  to  think . E-mail  is,  I always  can  only  think  about 
it  at  this  time. 

Here  Hiroko  pointed  out  the  difference  between  e-mail  dialogue  journals  and  paper  dialogue  journals:  unplanned  vs. 
planned.  E-mail  writing,  like  a telephone  or  face-to-face  conversation,  is  spontaneous 

Implicationi 

The  results  of  this  study  are  not  conclusive.  The  sample  of  the  study  was  obviously  limited.  However,  this 
project  was  a success  and  suggests  important  implications  in  utilizing  e-mail  as  a medium  to  conduct  dialogue  journaling 
in  ESL  classrooms. 

E-mail  provides  an  alternative  tool  for  students  to  use  for  dialogue  journaling.  While  it  is  not  appropriate  to 
assume  that  e-mail  is  the  tool  that  should  be  used  by  every  student  in  dialogue  journaling,  the  unique  features  of  e-mail 
do  appeal  to  some  students  and  thus  offer  them  a choice.  E-mail  is  a handy  tool  for  students  who  have  difficulties  in 
handwriting  and  who  are  interested  in  using  computers. 

E-mail  has  evident  advantages  over  paper  and  pencil  when  dialogue  journaling  is  conducted  outside  of  class. 
Many  ESL  teachers  conduct  dialogue  journaling  during  class  time.  However,  with  limited  class  time  to  cover  course 
materials,  teachers  might  find  it  difficult  to  devote  15-20  minutes  to  dialogue  journaling  each  day.  E-mail  provides  an 
ideai  environment  for  dialogue  journaling  outside  of  classroom.  It  eliminates  the  problems  of  having  to  coordinate  a 
communication  time  and  the  hassles  of  having  to  deal  with  paper  work. 

E-mail  can  also  play  a role  in  changing  inappropriate  student  writing  habits.  Student  writers,  especially  ESL 
beginning  writers  tend  to  edit  their  writing  prematurely  in  their  writing  process.  The  present  study  found  out  that  this 
problem  persisted  even  when  the  students  were  doing  dialogue  journaling,  which  is  a type  of  freewriting  practice.  E-mail 
provides  a writing  environment  whicn  encourages  students  to  put  down  their  thinking  as  fast  as  possible  without 
spending  too  much  time  agonizing  on  words,  phrases,  grammar  and  sentence  structures.  ESL  students  can  use  e-mail  to 
get  used  to  freewriting  and  practice  their  writing  fluency,  by  just  focusing  on  communication. 

E-mail  dialogue  journaling  in  the  present  study  was  conducted  between  the  instructor  and  the  student.  However, 
the  unique  features  of  e-mail  can  easily  expand  the  dialogue  journal  beyond  the  range  of  the  student  and  the  instructor. 
During  the  interviews,  the  students  in  the  present  study  expressed  a strong  desire  to  communicate  with  each  other  via  e- 
mail.  In  doing  e-mail  dialogue  journaling,  the  students  can  be  paired  with  other  students  at  a higher  language  level.  The 
students  also  can  be  paired  with  other  competent  language  users  on  or  off  the  campus.  In  this  way,  e-mail  offers  the 
opportunity  for  ESL  students  to  interact  with  more  people  with  whom  they  are  interested  and  who  can  help  them  in  their 
written  language  acquisition. 

The  discussion  of  the  potential  of  e-mail  in  writing  must  also  take  into  consideration  the  continuing  rapid 
changes  in  computer  technology.  Voice  input  to  computers  is  just  now  becoming  available  at  a price  that  an  ESL  lab 
can  afford.  Such  voice  input  can  be  used  to  produce  a text  message  that  can  then  be  sent  via  e-mail  system,  When  the 
students  can  actually  speak  to  the  computer  and  the  computer  types  the  message  for  the  students,  when  the  students  can 
not  only  read  the  message,  but  also  be  able  to  hear  the  message,  how  is  that  going  to  affect  the  production  of  written 
language? 
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Stotsky  (1983)  postulates  that  the  relationship  among  the  language  skills  is  not  only  reciprocal,  but  also 
multidirectional.  She  considered  that  all  four  language  skills  (reading,  writing,  speaking  and  listening)  influence  each 
other  and  enhance  each  other.  If  this  theory  is  valid,  we  can  expect  that  the  availability  of  voice  input  in  an  e-mail 
system  will  not  only  enhance  the  development  of  the  writing  skills  of  ESL  students,  but  will  also  enhance  the 
development  of  the  speaking  and  listening  abilities  of  ESL  students. 

Guistein  (1983)  stated  that  the  fluent  writer  uses  language  functions  appropriately  in  a wide  range  of  contexts. 
E-mail  provides  students  another  context  in  which  they  need  to  achieve  competence.  E-mail  is  an  important 
communication  medium  in  the  real  world.  Students  should  be  prepared  for  an  e-mail  communication  environment.  "We 
need  to  introduce  them  to  e-mail  as  an  aspect  of  the  work  environment"  (Hawisher  & Moran,  1 993,  p.  638). 

At  the  present  time,  the  use  of  e-mail  is  increasing  at  a speed  of  over  300,000%  a year  (Stonehill,  1993), 
however,  "the  acceptance  of  e-mail  in  academic  setting  has  been  less  than  lukewarm"  (D’ouza,  1991,  p.  106).  In  the  field 
of  composition,  "electronic  mail  has  received  scant  attention"  (Hawisher  & Moran,  1993,  p.  627).  E-mail  is  largely  used 
as  a personal  communication  tool  among  faculty  members  and  researchers.  The  function  and  impact  of  e-mail  as  an 
instructional  tool  have  not  been  adequately  explored.  The  present  study  suggests  that  e-mail  is  a valuable  tool  in  a 
writing  class.  As  Hawisher  and  Moran  (1993,  p.  638)  pointed  out:  "We  writing  teachers  need  to  pay  attention  to  e-mail 
in  our  writing  classes  because  it  is  here,  because  it  is  a means  of  written  communication,  and  because  we  as  academics 
are  using  it.  E-mail,  we  believe,  deserves  a place  in  the  curriculum.” 
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INTRODUCTION 


PURPOSE  OF  STUDY 

The  study  of  cognitive  styles  is  not  new  in  the  field  of  educational  psychology.  The  earliest  recorded  research 
was  conducted  at  the  turn  of  the  century  by  German  psychologists  (Coop,  1971).  Over  3,000  research  studies  to  date 
have  been  conducted  in  the  field  of  cognitive  psychology.  According  to  Chinien  and  Boutin  (1992-93),  cognitive  styles 
are  defined  as  the  “information  processing  habits  representing  the  learner’s  typical  mode  of  perceiving,  thinking,  problem 
solving,  and  remembering.”  Cognitive  style  is  concerned  with  how  an  individual  processes  information.  This  includes 
any  process  which  acquires  knowledge  (e.g.,  memory,  perception,  thought,  and/or  problem  solving).  Cognitive  styles 
are  not  likely  to  change  with  time  or  with  training  (Ausbum  & Ausbum,  1978  and  Witkin,  Oltman,  Raskin,  & Karp, 
1971).  Technology,  information  systems,  and  computer  based  training  should,  therefore,  be  tailored  to  the  individual 
most  likely  to  utilize  that  system.  Conversely,  if  systems  are  already  in  place  and  cannot  be  easily  adapted  or  changed, 
individuals  who  possess  the  appropriate  cognitive  abilities  should  be  selected  to  use  these  systems.  In  a 1994  study, 
Lyons-Lawrence  stated  that  materials  that  “stimulate  one  (individual)  may  be  confusing,  distracting,  and  difficult  for 
another,  and  too  simple  for  a third.”  Individuals  with  various  learning  styles  differ  in  their  performance  when  using 
technology.  For  example,  working  with  computers  may  be  unsuccessful  or  unproductive  if  the  individual  is  not  visually 
oriented  (Lyons-Lawrence,  1994). 

Cognitive  style  is  also  not  a single  entity.  Most  educational  psychologists  recognize  nine  to  eleven  major 
dimensions  of  cognitive  style  including:  scanning;  categorizing;  reflectivity/impulsivity;  and  field  independence/field 
dependence  (Whyte,  1990  and  Whyte,  Knirk,  Casey,  & Willard  1990-91). 

Although  many  cognitive  styles  have  been  identified  and  researched,  there  have  been  very  few  studies  which  have 
sought  to  correlate  different  measures  of  cognitive  style  into  a “multidimensional  construct”  (Kini,  1994).  Since  there 
would  be  a clear  advantage  to  identifying  such  a construct  we  attempted  to  map  the  “typical”  USAF  leader  of  today  and 
tomorrow  by  seeking  to  establish  a cognitive  map  of  the  students  and  Field  Grade  officers  (i.e.,  the  ranks  of  Major  and 
Lieutenant  Colonel)  in  the  Air  Force  today. 


COGNITIVE  STYLE 

Cognitive  Style  Defined 

Messick  (1976)  defines  cognitive  learning  styles  in  the  following  quote;  “Cognitive  learning  style  helps  explain 
how  an  individual  responds  to  a wide  range  of  intellectual  and  perceptual  stimuli.  Each  person’s  style  is  determined  by 
the  way  he  takes  note  of  his  total  surroundings.. .how  he  seeks  meaning,  how  he  becomes  informed.  Messick  (1984) 
contains  perhaps  the  most  complete  discussion  of  cognitive  style  available.  In  his  article,  Messick  defines  cognitive 
style  as  "characteristic  self-consistencies  in  information  processing  that  develop  in  congenial  ways  around  underlying 
personality  trends."  He  also  states  that  cognitive  styles  are  "intimately  inter-woven"  with  motivational,  affective  and 
temperamental  structures  to  produce  total  personality.  Perhaps  the  most  prolific  writer  in  the  area  of  cognitive  styles  was 
Herman  A.  Witkin.  As  part  of  their  extensive  research,  Witkin,  Moore,  Goodenough  and  Cox  (1977)  determined  that 
cognitive  styles:  1)  deal  with  the  "form"  of  cognitive  activity,  not  its  content  (e.g.,  thinking,  perceiving,  problem 
solving,  etc.);  2)  are  "pervasive  dimensions"  in  that  they  are  a feature  of  not  only  personality  but  also  cognition;  3)  are 
stable  over  time;  and  4)  are  also  bipolar.  In  other  words,  being  on  one  end  of  a cognitive  style  dimension  may  be  useful 
in  some  circumstances  while  not  in  others.  This  aspect  is  in  contrast  to  intelligence,  for  example,  where  "more”  is 
always  "better.” 

It  is  also  significant  to  note  that  there  appears  to  be  a negligible  relationship  between  cognitive  style  and  general 
intellectual  ability.  (Ausbum  & Ausbum,  1978  and  Witkin,  Oltman,  Raskin,  & Karp,  1971).  Ausbum  (1978)  notes  that 
these  relationships  are  extremely  negligible  and  fall  short  of  statistical  significance.  Although  the  relationship  between 
IQ  and  cognitive  style  appears  to  be  questionable  many  researchers  encourage  controlling  for  intelligence  when 
investigating  interactions  between  cognitive  styles  and  other  v.iriables  (Ausbum  & Ausbum,  1978;  Rosenberg,  Mintz  & 
Clark,  1977;  and  Wachtel,  1972). 

Related  to  this  research,  Witkin,  Moore,  Oltman,  Goodenough,  Friedman,  Owen  and  Raskin  (1977)  determined 
that  there  is  very  little  relationship  between  overall  college  achievement  (i.e.,  grade  point  average)  and  cognitive  style. 
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Studies  have  shown,  however,  relationships  of  cognitive  styles  and  performance  in  specific  subject  areas.  For  example, 
field  independents  tend  to  do  better  academically  in  math,  science  and  engineering. 

Cognitive  style  is  also  not  a single  entity.  Jonassen  and  Grabowski  (1993)  list  twelve  different  individual 
cognitive  styles/controls  including:  1)  Reflectivity/Impulsivity;  2)  Focal  Attention  (Scanning/Focusing);  3) 
Serialistic/Holistic;  4)  Field  Independence/Field  Dependence;  5)  Flexibility  (Constricted/Flexible);  6)  Category  Width 
(Narrow/Wide);  7)  Automization  (Strong/Weak);  8)  Visual/Haptic;  9)  Visualizer/Verbalizer;  10)  Leveling/Sharpening; 

1 1)  Analytical/Relational;  and  12)  Complexity/Simplicity. 

Implications  of  Cognitive  Style  on  Computer  Utilization 

In  their  1994  article  dealing  with  “Attributes  Affecting  Computer-Aided  Decision  Making,”  Moldafsky  and 
Kwon  state  that,  cognitive  style  refers  to  the  characteristic  processes  individuals  exhibit  in  the  acquisition,  analysis, 
evaluation,  and  interpretation  of  data  used  in  decision  making  and  has  been  shown  to  influence  a decision  maker’s 
evaluation  of  an  unstructured,  strategic  planning  problem.  Recent  research  has  shown  a strong  link  between  an 
individual’s  cognitive  style  and  their  reactions  to  computer  assisted  instruction  (CAI)  or  computers  in  general.  According 
to  Moldafsky  & Kwon  (1994),  research  indicates  that  cognitive  style  can  be  responsible  for  an  individual’s  skill  in 
information  processing,  decision-making  attitudes  toward  computers,  and  computer  anxiety.  In  1994,  Hsu,  Frederick,  and 
Chung  found  that  individuals  with  particular  cognitive  styles  significantly  outperformed  others  in  the  recall  of  the  content 
of  computer  based  instruction.  Rowland  and  Stuessy  (1988)  is  an  example  of  a study  which  matched  alternative  modes  of 
CAI  to  cognitive  style.  They  found  that  cognitive  style,  in  this  case  holists  and  serialists,  interacted  with  various  modes 
of  CAI  to  influence  student  achievement.  Burger  (1985)  further  supports  this  notion.  Her  research  is  another  example  of 
a study  which  investigated  the  interaction  of  this  particular  cognitive  style  (i.e.,  field  independence/field  dependence)  and 
preference  for  and  academic  achievement  in  computer  assisted  instruction. 

Cognitive  Style  and  Information  Processing 

To  provide  one  clear  example  of  the  differences  cognitive  style  can  make  between  individuals  in  information 
processing,  at  this  point  we  will  specifically  use  Field  Dependence/Independence,  to  describe  researched  processes,  since  it 
is  the  most  widely  researched  style  we  included  in  this  study.  Witkin,  1977,  found  that  individuals  with  various 
cognitive  styles,  in  this  case  field  independents  (FIs),  are  able  to  precisely  identify  the  critical  information  contained  in  a 
complex  visual  environment.  Field  dependents  (FDs),  on  the  other  hand,  generally  do  not  mentally  restructure  a visual 
presentation.  FDs  accept  and  interact  with  information  the  way  it  is  presented  (Witkin,  1977  and  Dwyer  & Moore, 

1992).  These  facts  have  strong  implications  for  intelligence  gathering,  information  processing,  and  the  critical  analysis 
of  visual  information.  Flannery,  1993,  found  that  FDs  process  information  in  a “simultaneous  manner.”  Ideas  are  seen 
all  at  once  rather  than  in  some  observable  order.  If  information  is  not  connected  to  something  the  individual  values,  it 
(the  information)  is  discarded.  FIs  use  logical,  inductive,  information  processing.  They  perceive  information 
objectively.  Information  for  FIs  does  not  have  to  be  concrete  or  personalized  (Flannery,  1993). 

PROJECT  DESIGN/PARTICIPANTS/MATERIALS 

Project  Design 

In  order  to  attempt  to  correlate  different  measures  of  cognitive  style  into  a “multidimensional  construct”,  we 
decided  to  utilize  three  existing  test  measurements:  Gestalt  Completion  Test,  testing  Cognitive  Flexibility;  Kolb 
Learning  Styles  Inventory,  evaluates  the  way  individuals  learn  and  deal  with  ideas  and  information  in  day-to-day 
situations;  and  Group  Embedded  Figures  Test  (GEFT),  which  determines  field  independence/dependence.  These  tests  were 
selected  due  to  their  reliability  factors  and  to  provide  the  best  description  of  the  “typical”  United  States  Air  Force  officer 
cognitive  style. 

Participants 

Selection  of  participants  used  in  the  study  came  exclusively  from  the  United  States  Air  Force  Academy.  Current 
officers  at  the  Major,  and  Lieutenant  Colonel  level  volunteered  to  participate  in  this  study  and  thus  represent  the  officers 
of  today  in  this  study.  They  were  recruited  from  across  educational  disciplines,  backgrounds  and  career  fields. 

Information  collected  about  each  officer  participant  included  gender,  age,  rank,  number  of  years  of  school  completed, 
undergraduate  major,  graduate  major,  last  degree  granted,  number  of  courses  completed  in  computer  science,  source  of 
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commissioning,  primary  AFSC/Career  Field,  amount  of  Professional  Military  Education  completed,  ethnic  group  and 
nationality. 

To  represent  the  Air  Force  officer  of  tomorrow,  we  surveyed  the  current  cadet  population  at  the  Air  Force 
Academy.  We  randomly  selected  senior  and  junior  undergraduate  level  students.  These  students  included  cadets  in 
leadership  and  non-leadership  positions,  randomly  placed  by  the  registrar  in  class  sections  of  core  Military  Art  and 
Science  courses.  We  collected  information  from  each  cadet  participant  on  gender,  date  of  birth,  class  level,  number  of 
courses,  completed  in  computer  science,  ethnic  group,  nationality,  and  academic  major. 

The  three  cognitive  styles  tests  were  administered  to  all  participants  in  classroom  environments.  They  were 
typically  all  administered  at  one  sitting.  On  occasion  the  GEFT  tests  were  administered  in  one  class  session  and  the 
other  two  tests  in  a different  class  session  due  to  class  time  availability.  This  is  not  seen  by  the  researchers  as  a 
confounding  factor  since  the  tests  do  not  build  upon  each  other  in  any  way.  All  tests  were  collected,  scored,  and  data 
compiled  through  the  use  of  SPSS. 


TESTING  MATERIALS 

GROUP  EMBEDDED  FIGURES  TEST  (GEFT) 

The  Group  Embedded  Figures  Test  (GEFT),  measures  the  cognitive  style  dimension  of  field  independence  and 
field  dependence.  The  GEFT  consists  of  18  complex  figures.  Individuals  must  find  a simple  embedded  geometric  figure 
which  is  hidden  in  a complex  one  (Donlon,  1977;  Thompson  & Meloncon,  1987;  Willard,  1985  and  Witkin,  Oltman, 
Raskin  & Karp,  1971).  As  in  previous  studies,  students  who  fall  “at  or  above  the  third  quartile''  will  be  designated  as 
field  independent.  Field  dependents  are  those  “whose  scores  on  the  GEFT  fall  at  or  below  the  point  that  represents  the 
first  quartile.”  The  reliability  of  this  test  is  .82  for  both  males  and  females  (Canino,  1988  and  Whyte,  1990). 

Overview  of  Field  Independence/Field  Dependence 

According  to  Rasinski  (1983),  field  independence/field  dependence  is  "by  far"  the  most  researched  of  all  cognitive 
styles.  Most  research  in  this  area  began  in  the  early  1950's  and  1960's  (Witkin,  Lewis,  Hertzman,  Machover,  Bretnall 
Meissner  & Wapner,  1954  and  Karp,  1963).  Herman  A.  Witkin,  Donald  R.  Goodenough  and  Philip  K.  Oltman  have 
produced  most  of  the  substantive  research  in  this  area  in  the  last  30  years  (Bertini,  Pizzamiglio,  & Wapner,  1986  and 
Witkin  & Goodenough,  1981).  Goodenough  (1976)  defines  field  independent  individuals  known  as  analytic  or  articulate, 
and  field  dependent  individuals  known  as  global. 

According  to  Willard  (1985),  this  dimension  is  concerned  with  an  individual's  ability  to  "perceive  a part  of  a 
stimulus  as  discrete  from  its  surroundings  through  active  and  analytic  as  opposed  to  passive  and  global  processes."  It  is 
also  significant  to  note  that  the  field  independence/field  dependence  dimension  is  "bipolar."  According  to  Witkin  and 
Goodenough  (1977),  "each  of  the  contrasting  cognitive  styles  has  components  that  are  adaptive  to  particular  situations, 
making  the  dimension  value  neutral." 

Personality  Traits  oL Field  ladependents/Pependents 

In  an  early  work,  Witkin,  Lewis,  Hertzman,  Machover,  Bretnall  Meissner,  and  Wapner  (1954)  described  a field 
independent  individual  as  almost  the  complete  opposite  of  a field  dependent.  A field  independent  is  analytical, 
independent,  and  can  function  with  very  little  environmental  support.  He/she  also  tends  to  have  low  anxiety  and  a high 
self-image.  A field  dependent,  on  the  other  hand,  is  passive,  shows  a lack  of  initiative  and  has  a readiness  to  submit  to 
authority.  Although  field  dependents  tend  to  be  warm  and  likable  they  generally  have  low  self-images. 

In  a follow-up  book,  Witkin  and  Goodenough  (1981)  found  that  based  on  a field  independent's  personality 
characteristics  they  also  tend  to  keep  to  themselves  or  desire  to  work  alone.  They  are  sometimes  viewed,  as  a result,  as 
inconsiderate,  distant  and  demanding.  Field  dependents  generally  make  greater  use  of  information  from  other  students 
(Willard,  1985.) 


■Specific  Characteristics  of  Field  Independence/Pependvncs 


Field  Independence 

Canino  and  Cicchelli  (1988)  define  field  independent  individuals  as  those  who  are  capable  of  perceiving  items  as 
discrete  from  background  or  field.  They  also  learn  better  when  they  are  allowed  to  develop  their  own  strategies  in 
problem-solving  nonsocial  domains.  As  part  of  their  extensive  research,  Witkin  and  Goodenough  (1977)  stated  that  field 
independents:  prefer  solitary  activities;  are  individualistic;  are  cold  and  distant  in  relations  with  others,  are  aloof,  never 
feel  like  embracing  the  whole  worl;  are  not  interested  in  humanitarian  activities;  value  cognitive  pursuits;  are  concerned 
with  philosopical  problems;  are  task  oriented;  have  work  oriented  values  such  as  efficiency,  control,  competence,  and 
excelling. 

Extensive  amounts  of  research  which  delineate  various  attributes  of  field  independent  individuals  have  been  compiled 
(Goodenough,  1976;  Guerrieri,  1978;  Malancon  & Thompson,  1990;  Mikos,  1980).  The  following  is  a compilation  of 
pertinent  research; 

1.  Field  independent  (FI)  elementary  students  gain  conservation  ability  more  quickly  than  FD  counterparts  (Wicker, 

1980). 

2.  FI's  seem  to  be  able  to  learn  concepts  more  efficiently  (Stasz,  Shavelson,  Cox,  & Moore,  1976). 

3.  FI's  have  better  reading  comprehension  skills  (Pitts  & Thompson,  1984;  Rasinski,  1983;  and  Spiro  & Tirre, 

1979) . 

4.  It  was  reported  that  high  scores  on  Witkin's  tests  (i.e.,  FI)  indicate  high  spacial  and  artistic  abilities  (Mayo  & 

Bell,  1972). 

5.  FI's  tend  to  favor  analytical  professions  (e.g.,  experimental  psychology,  mathematics,  surgical  nursing) 
(Rosenberg,  Mintz  & Clark,  1977). 

6.  A significant  number  of  engineers  are  field  independent  (Barrett  & Thornton,  1967). 

7.  FI's  are  likely  to  make  the  following  vocational  choices:  science  and  industrial  arts  teachers;  and  production 

managers  (Witkin,  Moore,  Oltman,  Goodenough,  Friedman,  Owen  & Raskin,  1977). 

8.  Field  independence  is  linked  to  academic  achievement  (e.g.,  passing  the  GED  exam)  (Donnarumma,  Cox  & Beder, 

1980) . 

Field  Dependence 

Field  dependent  individuals  are  almost  complete  opposites  of  their  counterparts.  Canino  and  Cicchelli  (1988) 
define  field  dependent  learners  as  those  who  are  not  as  able  to  separate  "elements  from  their  surroundings."  They 
experience  their  environment  more  globally  and  usually  accept  the  organization  provided  by  the  perceptual  field.  FDs 
also  prefer  to  interact  with  a teacher  and  tend  to  learn  better  with  structure.  Witkin  and  Goodenough  (1977)  stated  that 
field  dependents:  like  being  with  others,  are  sociable,  are  gregarious,  are  affiliation  oriented,  are  socially  outgoing,  prefer 
interpersonal  and  group  to  intrapersonal  circumstances,  seel$  relations  with  others  and  show  need  for  friendships,  show 
participativeness,  are  interested  in  people  artd  want  to  help  others,  and  now  many  people  and  are  known  to  many  people. 

As  stated  above,  extensive  research  has  been  compiled  which  delineates  various  attributes  of  the  field  dependent  dimension 
(Goodenough,  1976;  Guerrieri,  1978;  Malancon  & Thompson,  1990;  and  Mikos,  1980).  The  following  is  a compilation 
of  pertinent  research: 

1 . Field  dependents  show  a much  greater  preference  for  working  physically  closer  to  others  than  FIs  (Witkin  & 

Goodenough,  1977). 

2.  Field  dependent  people  also  not  only  prefer  more  to  be  with  people  but  also  tend  to  be  more  popular  with  peers 

(Malancon  & Thompson  (1990.) 

3.  FDs  tend  to  favor  the  vocational  choices  of  social  studies  and  elementary  school  teachers  (Witkin,  Moore, 

Oltman,  Goodenough,  Friedman,  Owen  & Raskin,  1977). 

4.  Field  dependents  are  also  attracted  to  ’’interpersonal"  occupations  (e.g.,  social  work,  psychiatric  nursing  and 

clinical  psychology)  (Rosenberg,  Mintz  & Clark,  1977). 
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GESTALT  COMPLETION  TEST 

The  Gestalt  Completion  Test  measures  cognitive  flexibility  and  is  one  of  72  factor-referenced  cognitive  tests 
provided  by  the  Education  Testing  Service  and  a part  of  the  Kit  of  Factor-Referenced  Cognitive  Tests  (1976).  This  test 
specifically  measures  the  speed  of  closure  which  is  defined  as  the  ability  to  write  an  apparently  disparate  perceptual  field 
into  a single  concept  (Ekstrom,  French,  Harman  and  Dermen,  1976).  These  same  authors  identified  that,  “According  to 
Carroll  (1974),  speed  of  closure  ‘requires  a search  of  a long-term  memory  visual-representatinal  memory  store  for  a match 
for  a partially  degraded  stimulus  cue.’  Strategies  employed  may  include  utilizing  hypotheses  from  association  in  long- 
term memory  or  restructuring  the  stimulus  perception.”  This  test  requires  the  participant  to  discern  individual  pieces  of  a 
whole  and  identify  the  whole  figure  or  picture.  There  are  no  clues  provided  as  to  what  the  complex  figure  may  be.  No 
context  is  provided  for  reference  to  environment  or  cognitive  situation.  The  test  is  comprised  of  two  samples  to 
familiarize  participants  with  expectations.  The  remainder  of  the  test  is  divided  into  two  sections,  each  with  ten  boxes 
containing  incomplete  black  and  white  pictures.  Participants  are  asked  to  identify  the  objects  within  the  boxes  using 
only  their  imaginations.  There  are  two  minutes  allowed  to  complete  each  section.  Scores  run  on  a continuum  from  1 
through  20  with  1 representing  the  highly  constricted  learner  and  20  representing  the  highly  flexible  learner. 

As  identified  by  two  independent  testing  sources  provided  in  the  Kit  of  Factor-Referenced  Cognitive  Tests 
(1976),  the  reliability  of  the  Gestalt  Completion  test  among  college  aged  participants  varies  from  .85  to  .82.  This  high 
level  of  reliability  led  to  our  selection  of  this  cognitive  test  for  inclusion  is  this  study. 

Overview  of  Cognitive  Flexibility 

Cognitive  flexibility  has  been  studied  very  little  in  comparison  to  Field  Independence/dependence  or  Kolb’s 
Learning  Style  Inventory.  We  determined,  however,  that  because  of  its  supposed  relationship  to  FI/FD  that  it  must  be 
included  to  determine  a clearer  picture  of  the  cognitive  style  of  the  typical  Air  Force  Officer.  Characteristics  of  this  style 
imply  different  abilities  and  approaches  to  problem  solving  than  does  FI/FD.  Cognitive  Flexibility  as  identified  by 
Jonassen  and  Grabowski  (1993),  “..is  a measure  of  the  ability  to  ignore  distractions  in  order  to  focus  on  relevant  stimuli 
(Klein,  1954)..” 

The  identification  of  cognitive  flexibility  stems  from  clinical  research  accomplished  primarily  by  Gardner  and 
Klein  and  secondarily  by  some  of  their  colleagues  at  the  Menninger  Clinic.  They  were  primarily  researching  the  concept 
of  how  cognitive  structures  modulate  and  control  human  drives.  The  distinct  difference  in  their  research  was  directed 
toward  the  element  that  personality  plays  in  determining  cognitive  roles.  This  element  was  defined  by  Klein  as  cognitive 
style,  which  also  contains  the  components  of  emotion,  motivation  and  affective  elements  (Ludwig  & Lazarius,  1983). 

Specific  Characteristics  of  Cognitive  Flexibiltv 

Flexibility 

Flexible  processors,  according  to  Jonassen  and  Grabowski  (1993),  are:  focused,  analytical,  open  to  change,  use 
feelings  and  other  emotions,  use  internal  cues  as  information  sources,  use  all  available  external  cues. 

Constricted 

Constricted  processors  according  to  the  same  sources  display  characteristics  opposite  to  those  of  Flexible 
individuals.  Those  characteristics  are  described  as:  distracted,  global,  resistant  to  change,  avoid  feelings  or  emotion,  use 
reaction  as  information  sources,  over  generalize  cognitive  set  cues. 

Recent  research  involving  cognitive  flexibility  is  looking  at  Hypermedia  and  the  application  of  knowledge  in 
new  situations,  comprehension,  problem  solving,  and  decision  making.  Findings  in  these  studies,  however,  are  not  yet 
generalizable  to  this  study.  Reference  research  will  continue  to  be  made  to  identify  any  computer  skill  attributions  and 
related  findings  concerning  this  cognitive  style  which  may  impact  future  implications. 

KOLB  LEARNING  STYLES  INVENTORY  II  *85  (LSI  1985) 

Kolb’s  Learning  Style  Inventory  II  ‘85  is  a tool  recognized  in  the  education  community  for  the  measurement  of 
an  individual’s  intrinsic  learning  style  or  individuals  predisposition  in  any  given  learning  situation.  It  was  developed 
according  to  the  theory  of  learning  expressed  in  the  Experiential  Learning  Model  (ELM)  (Kolb,  1974;  Kolb,  1976;  Kolb, 


787 


01 


Rubin  and  McIntyre,  1974).  It  was  designed  to  be  utilized  specifically  with  an  adult  population  in  education  or 
employment  settings.  In  1985  the  Learning  Style  Inventory  underwent  a revision  intended  by  David  A.  Kolb  as 
“improvements  designed  to  enhance  the  scientific  measurement  specifications  and  the  inventory  s practical  uses  in 
education  and  counseling”  (Technical  Specifications.  1985).  The  LSI  1985  is  a set  of  12  completion  statements  with  4 
rank  ordered  endings  requiring  approximately  10  minutes  to  complete  and  another  5 to  10  minutes  to  self-score. 

Reliability  and  Validity 

Technical  Specifications  (1985)  reports  very  good  internal  consistency  coefficients  as  measured  by  Cronbach’s  a 
coefficients  (ranging  from  .82  for  Concrete  Experience  (CE)  and  .83  for  Abstract  Conceptualization  (AC)  to  .78  for 
Active  Experimentation  (AE)  and  .73  for  both  Reflective  Observation  (RO).  Tukey’s  test  measured  an  almost  perfect 
additivity  of  1.0  (Technical  Specification.  1985.)  Similar  data  on  internal  consistencies  or  reliabilities  were  reiterated  in 
subsequent  studies  by  Sims,  Veres  III,  Watson,  and  Buckner  (1986)  Sims,  Veres  III,  and  Shake  (1989). 

Validity  is  reflected  in  the  intercorrelations  among  the  mode  of  leming  style  and  difference  scores  on  the  (AC- 
CE)  and  (AE-RO)  bipolar  dimensions.  As  expected,  these  intercorrelations  are  in  the  negative  direction  although  they 
vary  greatly  in  magnitude,  the  ranges  were  reported  from  -.05  to  -.85  with  an  absolute  value  mean  of  .36. 

These  findings  are  substantiated  by  Cornwell  and  Manfredo  (1994).  Their  findings  tended  to  support  Kolb’s 
generalizations  about  the  relationship  of  any  two  learning  orientations  to  a respective  learning  style. 

1.  (AE)  “doing”  and  (CE)  “feeling”  to  Accomodators 

2.  (RO)  “watching”  and  (CE)  “feeling”  to  Divergers 

3.  (RO)  “watching”  and  (AC)  “thinking  to  Assimilators 

4.  (AE)  “doing”  and  (AC)  “thinking”  to  Convergers 

Cornwell  et  al.  described  a “functional  relationship  existing  between  the  two  learning  style  typologies  such  that 
each  classification  of  learning  orientation  corresponds  to  only  two  classifications  of  the  learining  typology.  As  an 
example,  primary  learning  style  (PLS)  “feeling”  corresponds  to  learning  style  type  (LST)  diverger  or  accommodator. 
Similarly,  LST  accommodator  corresponds  to  PLS  “feeling”  and  “doing”  (1994). 

David  A.  Kolb  defines  learning  styles  as  an  individual’s  self-diagnosed,  preferences  in  the  perception  and 
subsequent  processing  of  information  (Kolb,  1984;  Jonassen  & Grabowski,  1993).  Crossing  the  perceptual  bi-polar 
continuum  of  concrete  experience  (CE)  versus  abstract  conceptualization  (AC)  with  the  information  transformational  bi- 
polar continuum  of  reflective  observation  (RO)  and  active  experiementation  (AE)  differentiates  four  types  of  learning 
styles. 

1 . Divergers  experience  their  environment  concretely  through  their  feelings  related  to  the  tangible  here  and  now 

(CE)  and  transform  it  through  internal  reflection  or  thought  (RO) 

2.  Assimilators  experience  their  world  symbolically  through  abstract  conceptualization  (AC)  and  transform  it 

through  thought  (RO)  as  the  Divergers. 

3.  Convergers  perceive  their  environment  through  analytic  thought  or  abstract  conceptualization  (AC)  and 

transform  that  information  through  action  (AE) 

4.  Accomodators  grasp  their  environment  concretely  through  their  feelings  (CE)  and  also  utilize  action  or  actively 

manipulate  their  environment  (AE)  to  transform  these  experiences  or  information  (Jonassen  & Grabowski,  1993; 

Krahe,  1993) 

Heredity,  previous  formal  or  informal  socialization  and  education  experiences,  as  well  as  the  immediate 
environment  play  a large  part  in  the  tendency  of  an  individual  to  favor  some  learning  abilities  over  others. 

Specific  Characteristics  of  Kolb’s  Learning  Styles 

Jonassen  and  Grabowski  (1993)  described  some  general  strengths  and  weaknesses  associated  with  each  of  the  four 
learning  styles  reflecting  the  bi-polar  nature  of  the  dimension  reflecting  the  individual’s  perception  of  information  in 
addition  to  how  that  individual  transforms  or  processes  that  information. 
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Divergers 

Divergers  have  the  ability  to  assimilate  or  synthesize  a wide-range  of  disparately  different  observations  into  a 
comprehensive  explanation.  This  enables  them  to  generate  many  ideas,  they  are  intuitive,  imaginative,  have  many  broad 
cultural  interest,  and  are  able  to  perceive  many  divergent  viewpoints,  their  people-oriented  skills  enable  them  to  relate 
well  to  others.  On  the  downside,  Di  vergers  are  “less  concerned  with  theories  or  generalizations”  (p.250).  they  approach 
situations  in  a less  thoughtful,  systematic  or  scientific  way.  This  inhibits  their  ability  to  make  decisions. 

Assimilators 

Assimilators  take  a focused,  systematic  and  scientific  approach  to  their  environment.  This  use  of  logic, 
inductive  reasoning  skills,  and  the  ability  to  view  multiple  perspectives  is  needed  to  theoretically  model  building.  This 
is  reflected  in  their  ability  to  organize  information  well  which  is  needed  in  the  design  of  stable  experiments. 

Assimilators  prefer  analytic,  abstract,  and  quantitative  task  and,  conversely,  feel  uncomfortable  performing  qualitative  or 
concrete  tasks.  They  focus  less  on  interpersonal,  people  oriented  skills  which  reduces  their  ability  to  impact  or  influence 
others.  Because  they  are  less  action  oriented,  they  are  less  able  to  apply  theories  and  model  to  the  real  world. 

Convergers 

Convergers  bring  a logical,  pragmatic,  as  well  as  focused  and  unemotional  perspective  to  any  situation.  They 
have  the  ability  to  problem  solve,  make  thoughtful  decisions,  and  get  the  job  done,  many  times,  creating  new  ways  of 
thinking  and  doing  when  the  situation  needs  it.  Their  focus  on  the  analytical  reduces  the  intuitive  understanding 
necessary  in  relation  to  people  skills  and  increases  their  discomfort  concerning  social  or  interpersonal  issues.  They  are 
sometimes  considered  non-artistic,  unimaginative  and  closed  minded  which  can  tend  toward  a narrow  range  of  interests. 
They  are  uncomfortable  with  the  qualitative  or  the  concrete.  They  are  more  concerned  with  the  “relative”  truths  than 
absolute  truths. 


Accomodators 

Accomodators  prefer  to  carry  out  plans  which  produce  action  and  results  based  on  facts  and  reality.  They  are  risk 
takers  and  enjoy  seeking  out  new  experiences.  This  enables  them  to  adapt  to  new  situations  and  environments  well. 

Their  op-minded,  intuitive,  people  oriented  approach  enables  them  to  influence  and  lead  others  or  impact  a situation 
because  of  their  personal  involvement.  Because  they  rely  more  heavily  on  other  people  for  information,  they  are  less 
scientific,  systematic,  and  analytical  in  their  approach  to  a given  situation,  they  tend  to  favor  a trial-and-error  approach  to 
their  environment  disregarding  theory.  They  are  sometimes  perceived  as  impatient  and  controlling.  Like  the  converger, 
they  are  less  concerned  with  absolute  truth  the  “relative”  truth. 


RESULTS/DATA  ANALYSIS 

RESULTS 

The  total  sample  size  equaled  107;  77  cadets,  30  officers.  Following  are  percentage  breakdowns  of  the  total 
population  for  gender,  nationality,  ethnic  group,  undergraduate  major  and  number  of  courses  in  computer  science. 


Gender: 


85%  male 
15%  female 


Nationality: 


Ethnic  Group: 


94.4%  American 
1 .9%  Other 
3.7%  Not  reported 

5.6%  African  American 
5.6%  Asian 
1 .9%  American  Indian 
1 .9%  Hispanic 
81.3%  Caucasian 
3.7%  Not  reported 
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Undergraduate  Major: 

2.8%  Behavioral  Science 

Computer  Science  Crs: 

7.5%  Biology 

12.1%  Business/Management 

9.3%  Computer  Science 

.9%  Economics 

35.5%  Engineering 

10.3%  Geography 

4.7%  History 

.9%  Languages 

2.8%  Math 

2.8%  Physics 

9.3%  Political  Science 

10.3%  Other 

1.9%  No  courses 
64.5%  1-2  courses 
17.8%  3-4  courses 
5.6%  5-6  courses 
5.6%  7-8  courses 
1.9%  9-10  courses 
2.8%  11+ courses 

These  statistics  identify  that  this  sample  population  mirrors  the  Air  Force  of  today  by  gender.  It  is 
representative  of  a primarily  American  population  with  diverse  ethnicity.  Fields  of  undergraduate  study  range  over  more 
than  twelve  different  subject  areas,  identifying  multiple  interests  and  expertise  throughout  the  group.  Finally,  these 
statistics  identify  that  this  population  is  familiar  with  computers  and  operations  of  computers  in  their  lives  with  98.1% 
having  taken  a minimum  of  1+  courses  in  computer  science. 

CADET  POPULATION 

The  cadet  population  closely  mimics  the  population  sample  as  a whole.  The  percentages  are  only  slightly 
different  but  emphasize  the  same  representative  population  over  all.  Following  are  the  specifics: 


Gender: 

90.9%  male 

9.1%  female 

Nationality: 

97.4%  American 
2.6%  Other 

Ethnic  Group: 

5.2%  African  American 
6.5%  Asian 
2.6%  American  Indian 
2.6%  Hispanic 
83.1%  Caucasian 

Undergraduate  Major: 

2.6%  Behavioral  Science 
5.2%  Biology 

15.6%  Business/Management 
9.1%  Computer  Science 

I . 3%  Economics 
40.3%  Engineering 
5.2%  History 
2.6%  Math 

6.5%  Political  Science 

II. 7%  Other 
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Computer  Science  Crs: 


74.0%  1-2  courses 
13.0%  3-4  courses 
5.2%  5-6  courses 
6.5%  7-8  courses 
1.3%  9-10  courses 

The  only  identifiable  differences  between  the  cadet  population  and  whole  sample  were  in  undergraduate  major, 
where  Physics,  Language  and  Geography  were  not  identified  as  a major  area  of  study.  Additionally,  no  students  had  taken 
above  1 1 computer  science  courses.  These  differences  are  not  considered  by  the  researchers  to  be  significant  since  the 
percentages  in  the  main  populations  for  these  features  were  minute. 

Consistency  with  the  total  population  is  significant.  Gender  representation  for  cadets  was  10%  female,  which  is 
in  direct  correlation  with  the  Air  Force  as  a whole.  97.2%  are  American,  with  primarily  a Caucasian  ethnicity,  but  with 
representation  from  multiple  groups.  Undergraduate  major  had  the  largest  percentage  in  Engineering,  but  a representation 
at  some  fairly  high  levels  appeared  in  9-f  major  areas  indicating  diverse  interests  and  expertise.  All  cadets  have  had  at 
least  one  to  two  courses  in  computer  science. 

Officer  Population 

Following  are  the  statistics  for  the  officer  population  in  comparison  to  the  total  sample. 


Gender: 

70%  male 
30%  female 

Nationality: 

86.7%  American 
13.3%  Not  reported 

Ethnic  Group: 

6.7%  African  American 
3.3%  Asian 
76.7%  Caucasian 
13.3%  Not  reported 

Undergraduate  Major: 

3.3%  Behavioral  Science 
13.3%  Biology 
3.3%  Business/Management 
10.0%  Computer  Science 
23.3%  Engineering 
3.3%  Geography 
3.3%  History 
3.3%  Languages 
3.3%  Math 

10.0%  Physics 
16.7%  Political  Science 

6.7%  Other 

Computer  Science  Crs:  6.7%  No  courses 

40.0%  1-2  courses 
30.0%  3-4  courses 
6.7%  5-6  courses 
3.3%  7-8  courses 
3.3%  9-10  courses 
10.0%  1 14*  courses 
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These,  again,  correlate  very  highly  with  the  percentages  of  the  total  sample.  The  officers  had  more  computer 
science  courses  than  the  sample  or  the  cadets.  Officers  had  a higher  population  of  females  who  participated.  The  ethnic 
diversity  was  not  as  broad,  but  still  represented  more  than  one  ethnic  group. 

In  addition  to  the  identified  categories  for  the  cadets  and  the  whole  sample,  the  following  data  was  collected 
specifically  for  officers.  This  was  to  identify  who  the  members  of  this  group  were  in  relation  to  the  total  Air  Force 
population. 


AFSC 

6.7%  Acquisition/Contracting 
3.3%  Air  Defense 
3.3%  Civil  Engineering 
10.0%  Communication 
6.7%  Engineering 
3.3%  Intelligence 
3.3%  Missiles 

10.0%  Navigator/Weapons  Systems 
20.0%  Pilot 
6.7%  Space  Operations 
6.7%  Other 
20.0%  Not  identified 

COMMISSIONING  SOURCE 

20.0%  AFROTC 
16.7%  OTS 
43.3%  USAFA 

GRADUATE  MAJORS 

3.3%  Behavioral  Science 
10.0%  Biology 
3.3%  Business/Management 
13.3%  Computer  Science 
6.7%  Education 
20.0%  Engineering 
3.3%  Geography 
3.3%  Languages 
10.0%  Physics 
13.3%  Political  Science 
13.3%  Other 

PROFESSIONAL  MILITARY 
EDUCATION 

33.3%  SOS 

36.7%  Intermediate  Service  School 
10.0%  Senior  Service  School 

RANK 

6.7%  0-3 
56.7%  0-4 
36.7%  0-5 

This  data  indicates  that  the  participating  population  does  represent  the  Air  Force  of  today.  Eleven*  AFSCs  are 
represented,  with  officers  from  all  commissioning  sources.  Graduate  majors  are  as  varied  as  the  undergraduate  majors.  A 
slightly  larger  Majors  participation  than  Lt.  Colonel  which  coincides  with  actual  numbers  of  officers  in  the  Air  Force  at 
those  levels  today. 

Because  of  the  congruence  between  cadets  and  officers  in  comparing  all  collected  data,  including  the  learning 
styles  results,  we  will  only  identify  the  findings  from  this  point  forward  for  the  group  as  a whole. 
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Following  are  the  results  for  each  individual  cognitive  style  tested  for  with  the  participants.  The  results  stand 
alone  and  reflect  scoring  of  the  tests  based  upon  requirements  of  each  testing  mechanism. 


FI/FD 

FIELD  DEPENDENCE/INDEPENDENCE 

Valid  Cum 


Value  Label 

Value 

Frequency 

Percent 

Percent 

Percent 

FI 

1 

92 

86.0 

86.0 

86.0 

>FI 

2 

7 

6.5 

6.5 

92.5 

>FD 

3 

2 

1.9 

1.9 

94.4 

FD 

4 

6 

5.6 

5.6 

100.0 

Total 

107.0 

100.0 

100.0 

Figure  2 


As  the  chart  (figure  1)  above  illustrates  this  population  is  86.9  extremely  field  independent.  The  combination  of 
FI  and  >FI  make  up  92.5%  of  the  population  with  only  7.5%  of  the  total  population  being  a combination  of  FD  and 
>FD,  identifying  a clear  tendency  for  Field  Independence. 


COGNITIVE 

FLEXIBILITY 

Valid 

Cum 

Value  Label 

Value 

Frequency 

Percent 

Percent 

Percent 

Extreme  Flexible 

1 

53 

49.5 

49.5 

49.5 

Flexible 

2 

51 

47.7 

47.7 

97.2 

Constricted 

3 

3 

2.8 

2.8 

100.0 

Total 

107.0 

100.0 

100.0 

Figure  3 


Since  the  tendency  for  cognitive  flexibility  is  expressed  along  a continuum  from  0 to  20,  this  team  of 
researchers  divided  the  continuum  into  four  quartiles: 

0-5  = extremely  constricted  6-10  = constricted 

11-15  = flexible  16-20  = extremely  flexible 


These  quartiie  identifiers  allowed  us  to  determine  to  what  extent  this  population  is  cognitively  flexible  or  constricted. 
The  chart  above  clearly  demonstrates  that  our  participants  are  97.2%  cognitive  flexible,  with  an  approximate  50-50  split 
between  extremely  flexible  and  flexible.  Only  three  participants  out  of  107  tested  constricted  with  0 testing  extremely 
constricted. 
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KOLB  LEARNING  STYLE 

INVENTORY 

Valid 

Cum 

Value  Label 

Value 

Frequency 

Percent 

Percent 

Percent 

Diverger 

1 

15 

14.0 

14.0 

14.0 

Assimilator 

2 

26 

24.3 

24.3 

38.3 

Converger 

3 

50 

46.7 

46.7 

85.0 

Accomodator 

4 

16 

15.0 

15.0 

100.0 

Total 

107 

100.0 

100.0 

Figure  4 


As  reflected  in  this  chart  46.7%  of  the  participants  tested  out  as  Convergers  with  the  next  largest  population 
being  Assimilators  at  24.3%. 


Comparison  of  Test  Material; 

As  a further  demonstration  of  the  statistics,  following  are  three  charts;  one  comparing  Cognitive  Flexibility  and 
FI/FD,  two  comparing  Kolb  Learning  Style  and  FI/FD,  and  the  third  comparing  Kolb  Learning  Style  and  Cognitive 
Flexibility.  A Chi-square  has  been  run  with  these  cross  tabs  to  determine  any  levels  of  significant. 


COGNITIVE  FLEXIBILITY  VS  FIELD  DEPENDENCE/INDEPENDENCE 


FLEXIBILITY 

Extreme  Flexible  Constricted 
Flexible  Row 


Count 

1 

2 

3 

Total 

% 

FI/FD  

FI  1 

47 

43 

3 

93 

86.9 

>FI  2 

2 

4 

6 

5.6 

>FD  3 

2 

2 

1.9 

FD  4 

2 

4 

6 

5.6 

Column 

Total 

53 

49.5 

51 

47.7 

3 

2.8 

107 

100.0 

Chi-Square  Value  DF  Significance 

Pearson  4.03688  6 .67168 

Likelihood  Ratio  5 . 1 254j 6 .52783 

Figure  5 
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KOLB  LEARNING  STYLE  VS  FIELD  DEPENDENCE/INDEPENDENCE 

KOLB  LRN  STYLE 

Diverger  Assimilator  Converger  Accomodator  Row 


Count 

1 

2 

3 

4 

Total 

% 

FI/FD 

FI 

1 

12 

25 

43 

13 

93 

86.9 

>FI 

2 

5 

1 

6 

5.6 

>FD 

3 

1 

1 

2 

1.9 

FD 

4 

2 

1 

2 

1 

6 

5.6 

Column 

Total 


Chi-Square 


Value 


Pearson  11.32612 

Likelihood  Ratio  12.93238 


15 

14.0 

DF 

9 

9 


26 

24.3 


50 
46.7 

Significance 

.25402 

,16569 

Figure  6 


16 

15.0 


107 


100.0 


KOLB  LEARNING  STYLE  VS  COGNITIVE  FLEXIBILITY 
KOLB  LRN  STYLE 

Diverger  Assimilator  Converger  Accomodator 


Count 

1 

2 

3 

4 

Row 

Total 

% 

FLEXIBILITY 

Extreme  Flexible  1 

9 

10 

23 

11 

53 

49.5 

Flexible  2 

6 

15 

25 

5 

51 

47.7 

Constricted  3 

1 

2 

3 

2.8 

Column 

15 

26 

50 

16 

107 

Total 

14.0 

24.3 

46.7 

15.0 

100.0 

Chi-Square  Value  DF  Significance 


Pearson  5.21428  6 .51664 

Likelihood  Ratio  6.05043  6 .41757 

Figure  7 

The  Air  Force  Officer  of  Today  and  Tomorrow 
Based  upon  the  data  collected  we  have  determined: 

1 . The  Air  Force  Officer  of  today  and  tomorrow  vary  little  in  their  statistical  makeup. 

2.  They  are  field  independent,  cognitively  flexible  and  convergers.  These  three  styles  have  common 
characteristics,  independently  defined,  as  supported  by  the  literature.  Common  characteristics  of  these  three 
styles  include 
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- Analytical , logical,  pragmatic 

- Concern  for  ideas  and  principles 

- Prefer  solitary  activities,  individualistic 

- Focused,  unemotional  perspective  to  situations 

- Self  motivated,  favoi  lectm  ; in  learning 

- Have  excellent  reading  comprehension  skills 

- Superior  problem  solvers,  adapting  the  approach  to  the  problem 

- Do  not  require  externally  provided  structure 

- Open  to  change 

- Creates  new  ways  of  thinking  and  doing  when  the  situation  requires 

- Work  oriented  values  such  as  efficiency,  control  competence,  and  excelling 

- Considered  non-artistic 

- Select  scientific,  experimental  psychology,  mathematics,  and  engineering  careers 

- Efficient  at  restructuring  and  organizing  visual/spatial  information 

- Adept  at  processing  unusually  complex  visual  materials 

DISCUSSION/IMPLICATIONS 

Based  upon  this  initial  research,  it  appears  that  it  may  be  possible  to  identify  a multidimensional  construct  for 
individuals.  These  researchers  believe  that  if  two  multidimensional  constructs  which  were  bipolar  could  be  identified, 
instruction  could  then  be  adapted  to  meet  the  needs  of  students  more  specifically.  With  the  extremes  identified,  the  needs 
of  the  individuals  leaning  in  either  direction  would  also  be  met.  While  we  believe  this  baseline  study  provides  a “first 
look”  at  previously  undocumented  information  about  comparison  of  three  learning  styles  inventories,  the  door  to  what 
this  means  has  only  been  slightly  opened. 

Future  studies  should  include  a larger  population  of  individuals  from  public  university  settings.  While  this 
group  was  highly  representative  in  many  areas,  it  did  not  represent  a normal  population  spread  for  gender  or  nationality. 

It  would  be  important  to  test  students  at  international  universites  to  see  if  a this  construct  could  be  generalized  to  a larger 
population. 

Only  three  tests  were  used  in  this  baseline  study.  We  must  expand  the  number  of  cognitive  tests  used  to  provide 
a more  extensive  picture  of  this  multidimensional  construct.  Styles  such  as  visual/verbalizer,  locus  of  control, 
impulsivity/reflectivity,  and  cognitive  width  would  provide  a more  in-depth  picture  of  our  population. 

We  must  look  at  development  of  a test  environment  which  mimics  the  real  world  to  see  how  individuals 
approach  problem  solving  and  decision  making.  We  should  track  such  items  as  cognitive  strategy  and  time  on  task, 
which  could  identify  ability  to  perform  in  stressful  situations. 

Clearly,  much  research  is  still  needed.  This  study  only  takes  a beginning  look  at  the  implications  of  cognition 
and  a multidimensional  construct.  Future  study  can  only  enhance  this  look  and  provide  necessary,  vital  approaches  to 
instructional  design  and  delivery. 
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In  the  wake  of  the  constructivist  movement  in  psychology  and  education,  the  field  of  instructional  design  (ID)  is 
reexamining  its  relationship  to  learning  and  instruction.  Many  ID  theorists  are  calling  for  more  situated  approaches  to 
the  design  of  instruction,  encouraging  teachers  and  local  groups  to  take  ownership  of  the  design  process  and  adapt  their 
methods  and  goals  to  the  needs  of  students  and  stakeholders  (e.g.,  Wilson,  Teslow,  and  Osman -Jouchoux,  1995).  Others 
have  defended  traditional  views  of  instructional  design  as  a prescriptive  science,  charged  with  developing  universal 
methods  and  strategies  that  will  result  in  effective,  efficient  instruction  (Merrill,  Drake,  Lacy,  & Pratt,  1996). 

The  problem  is  exacerbated  by  the  growing  negative  connotation  of 4 instruction.’  To  many  constructivist  educators, 
what  they  are  trying  to  accomplish  with  students  cannot  be  captured  by  ‘instruction.’  Instruction  is  typically  thought  to 
have  clear,  prespecified  learning  objectives,  teacher-determined  activities  and  instructional  strategies,  and  clear  boundaries 
in  time  and  space.  What  happens  when  learning  happens,  but  not  in  such  clearly  directed,  controlled  terms?  ID  theorists 
are  examining  alternate  metaphors  such  as  “learning  environments”  to  understand  and  describe  learning  where  the  learner 
assumes  more  direction  and  control  over  goals,  content,  and  methods  (e.g.,  Wilson,  1996). 

The  idea  of  “learning  communities”  has  also  been  discussed  as  an  alternative  metaphor  to  traditional  instruction.  What 
happens  when  groups  of  people  gather  together  to  provide  mutual  support  for  learning  and  performance?  How  would  that 
work?  Rather  than  being  controlled  by  a teacher  or  an  instructional  designer,  learners  might  “self-organize”  into 
functioning  communities  with  a general  goal  of  supporting  each  other  in  their  learning.  That  is  to  say,  the  function  of 
guidance  and  control  becomes  distributed  among  group  participants.  Specific  roles  of  group  members  are  not  assigned 
but  rather  emerge  from  the  interaction  of  the  whole. 

This  paper  is  our  initial  effort  to  outline  the  concept  of  a dynamic  learning  community  as  an  alternative  to  teacher- 
cont  oiled  or  pre-designed  instructional  systems.  We  argue  that  dynamic  learning  communities  constitute  an  important 
alternative  to  specifically  designed  instructional  systems,  and  that  communication  technologies  can  serve  to  support 
learning  communities  in  their  efforts.  We  present  below  an  outline  of  our  current  thinking.  For  future  papers,  we  intend 
to  gather  more  examples  or  case  reports  concerning  specific  learning  communities. 

What  is  a Dynamic  Learning  Community? 

Our  definition  for  DLCs  is  offered  in  Table  1.  We  see  DLCs  as  decentralized  groups  focused  and  interacting  enough  to 
form  a stable  community.  Let  us  first  unpack  the  elements  of  the  term: 

Communities.  Groups  become  communities  when  they  interact  with  each  other  and  stay  together  long  enough  to  form 
a set  of  habits  and  conventions,  and  when  they  come  to  depend  upon  each  other  for  the  accomplishment  of  certain  ends. 

Learning  communities.  In  truth,  all  communities  learn.  One  of  the  lessons  of  postmodernism  and  situated  cognition 
is  that  learning  cannot  be  separated  from  action.  We  are  learning  every  day,  in  everything  we  do.  We  add  the  qualifying 
term  to  our  definition  to  suggest  a community  sharing  a consensual  goal  to  support  each  other  in  learning.  Everybody 
expects  to  learn  and  is  prepared  to  engage  in  activities  at  least  partly  for  that  reason.  This  would  distinguish  learning 
communities  from  those  solely  concerned  with  entertainment,  political  action,  or  the  performance  of  an  immediate  task. 
We  would  note,  however,  that  groups  can  have  complex  agendas,  and  that  a group  may  have  multiple  goals  that  are 
commonly  shared  throughout  the  membership,  such  as  supporting  both  work  performance  and  learning  among  its 
members. 

Dynamic  learning  communities.  The  term  ‘dynamic’  is  added  to  distinguish  the  construct  from  traditional,  centralized 
groups  of  learners  found  in  many  classrooms.  In  a dynamic  community,  all  members  share  control,  and  everyone  learns, 
including  the  “teacher”  or  group  leader  (cf.  Wilson  & Cole,  1997;  Scardamalia  & Bereiter,  1994).  Transformative 
communication  is  the  norm,  wit  both  sender  and  receiver  of  messages  changed  by  the  interaction  (Pea,  1994).  Thus  a 
classroom  wherein  the  teacher  assigns  a project,  expecting  the  students  to  learn  something  but  not  expecting  herself  to 
learn — such  a classroom  would  not  yet  be  a dynamic  learning  community  because  all  participants  are  not  engaged  in  the 
learning  experience. 


Dynamic  learning  communities  (DLCs)  are  groups  of  people  who  form  a learning  community  generally  characterized 
by  the  following: 

— distributed  control; 

— commitment  to  the  generation  and  sharing  of  new  knowledge; 

— flexible  and  negotiated  learning  activities; 

— autonomous  community  members; 

— high  levels  of  dialogue,  interaction,  and  collaboration; 
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— a shared  goal,  problem,  or  project  that  brings  a common  focus  and  incentive  to  work  together. 

We  would  expected  to  see  the  following  additional  characteristics  over  time: 

Positive  Outcomes 

— capacity  to  adapt  to  local  conditions  and  evolve  over  time; 

— creativity  and  innovation; 

— crossing  of  traditional  disciplinary  and  conceptual  boundaries; 

— appreciation  of  diversity,  multiple  perspectives  and  epistemic  issues; 

— community  members  who  are  responsible  and  skilled  at  diagnosing  and  addressing  their  learning  needs; 

Negative  Outcomes 
— occasional  inefficiencies; 

— lack  of  control; 

— lack  of  predictability. 

Table  1 . Definition  and  characteristics  of  dynamic  learning  communities  (DLCs). 

DLCs  are  the  sort  of  open,  adaptive  system  described  by  Kevin  Kelly  in  his  book  Out  of  Control  (Kelly,  1994).  Open 
systems  are  defined  by  “( 1 ) the  absence  of  imposed  centralized  control;  (2)  the  autonomous  nature  of  subunits;  (3)  the 
high  connectivity  between  the  subunits,  and  (4)  the  webby  nonlinear  causality  of  peers  influencing  peers”  (Kelly  1994,  p. 
22,  reformatted). 

According  to  complexity  theorists  (see  also  Gleick,  1987;  Hayles,  1991;  Prigogine  & Stengers,  1984;  Waldrop,  1992), 
a complex,  adaptive  system  takes  on  life-life  qualities  such  as  intelligence,  intention ality,  self-correctability,  and  self- 
preservation.  Examples  of  complex  systems  include  the  stock  market,  ecological  systems,  and  living  organisms. 

Similar  qualities  of  intentionality  and  adaptivity  should  emerge  in  a learning  community  if  they  have  the  characteristics 
noted  in  Table  1 and  further  discussed  below. 

Distributed  control.  In  a typical  classroom,  the  teacher  is  in  charge.  The  teacher  makes  all  the  important  decisions, 
such  as  what  to  teach  and  how  to  teach  it.  In  a DLC,  nobody  is  in  control — everybody  is!  Conventions,  shared 
understandings,  rules  for  settling  disputes  or  for  governing  communications — All  of  these  are  negotiated  and  agreed  upon 
by  the  group  as  a whole.  So  are  learning  goals  and  methods.  If  one  member  has  a different  vision  for  where  the  group 
should  be  going,  this  is  presented  to  the  group  and  discussed.  The  community  is  bigger  than  any  single  member,  yet  it 
encompasses  the  perspectives  of  all  members. 

Commitment  to  the  generation  and  sharing  of  new  knowledge.  Everybody  learns.  Nobody  stands  apart,  pulling  the 
strings  for  the  sake  of  the  others.  By  sharing,  listening,  imitating,  and  watching,  all  members  of  the  learning 
community  benefit.  Those  with  greater  expertise  play  critical  roles  in  helping  and  modeling,  yet  they  are  expected  to 
learn,  solve  problems,  find  answers,  right  along  with  the  rest  of  the  group. 

Flexible  and  negotiated  learning  activities.  Specific  learning  goals  and  activities  largely  “happen.”  There  is  a sort  of 
natural  selection  of  activities — Those  that  are  successful  and  lead  to  learning  are  repeated  and  developed  and  shared,  while 
those  that  are  not  supported  by  the  group  fall  into  disuse.  This  can  lead  to  inefficiencies  and  a meandering  process  of 
development,  but  it  can  work. 

Autonomous  community  members.  One  of  Kelly’s  key  components  of  complex  systems  is  that  of  “autonomous 
agents”;  that  is,  community  members  must  have  a certain  room  to  direct  their  own  activities  and  make  decisions.  There 
needs  to  be  room  for  variation  and  differences  among  community  members;  otherwise  the  system  devolves  down  to  a 
single  controlled  perspective. 

High  levels  of  dialogue,  interaction,  and  collaboration.  High  levels  of  connectivity  are  essential  to  complex  systems 
and  to  DLCs  in  particular.  A neighbor  may  be  doing  great  things,  but  if  that  information  is  not  shared  via  constant 
communication,  then  other  community  members  will  not  be  aware  of  it.  Information  is  what  drives  the  feedback  loops 
that  lead  to  new  learning  and  change  in  the  overall  system. 

A common  focus  and  incentive  to  work  together.  DLCs  need  a reason  to  exist.  This  may  come  from  shared  interests, 
or  a common  goal,  problem,  or  project.  Outside  constraints,  such  as  market  or  job  demands,  may  provide  an  incentive 
for  a group  to  form.  Work  groups  may  be  motivated  to  keep  their  jobs.  The  free  market  of  Internet  listservs  may  allow 
communities  to  coalesce  around  very  localized  interests,  such  as  breeding  border  collies  or  following  the  fortunes  of  an 
NFL  franchise.  Public-school  students  may  converge  on  a project  or  major  assignment,  such  as  a yearbook  or 
newspaper.  Students  in  Carnegie-type  classrooms  may  be  hard-pressed  to  find  a threshold  of  common  purpose  sufficient 


to  create  a truly  dynamic  learning  community.  Certainly  such  a concept  competes  with  the  bulk  of  school  conventions 
and  cultural  forms. 

Consistent  with  other  complex  systems,  we  would  expect  to  see  a number  of  additional  features  emerge  in  DLCs, 
some  positive  and  some  negative.  These  are  briefly  discussed  below. 

POSITIVES 

Capacity  to  adapt  to  local  conditions  and  evolve  over  time.  Because  DLCs  depend  upon  each  member  for  information, 

DLCs  should  be  able  to  pick  up  on  changes  in  the  environment  more  quickly  than  controlled  instructional  groups. 

Behavior  of  DLCs  are  more  fluidly  defined  and  more  flexible  than  fixed-goal  and  fixed-strategy  systems,  allowing  easier 
adaptation  and  change  over  time. 

Creativity  and  innovation.  DLCs  will  tend  to  be  more  pluralistic  than  instructional  systems  because  behavior  is  not 
centrally  controlled.  Much  variant  behavior  may  prove  unfruitful,  but  in  amid  the  diversity,  some  ideas  will  show 
promise.  Thus  creativity,  change,  and  innovation  should  be  more  prevalent  than  in  instructional  systems.  The  DLC 
may  experience  more  failures,  but  more  innovative  successes  as  well. 

Crossing  of  traditional  disciplinary  and  conceptual  boundaries.  Typically,  a DLC  cares  less  about  its  disciplinary  base 

than  the  problem  it  is  trying  to  solve.  ^ Workers  from  a variety  of  backgrounds,  for  example,  may  form  a DLC  if  they 
face  a common  challenge  or  problem.  Each  person  brings  the  baggage  of  their  prior  experience,  and  each  submits  to 
being  influenced  by  the  community.  The  cross-fertilization  that  results  can  lead  to  new  categories  and  new  perspectives 
not  previously  perceived  by  established  communities. 

Appreciation  of  diversity,  multiple  perspectives , and  epistemic  issues.  Expertise  is  inherently  multi-perspectival  in  a 
DLC.  Members  come  to  respect  knowledge  that  comes  from  a variety  of  sources — people  of  different  backgrounds  and  R 

information  of  different  types.  Likewise,  community  members  develop  their  own  methods  for  testing  proposed 
knowledge  against  a variety  of  standards  and  codifying  that  knowledge  in  a way  that  can  be  shared  throughout  the  group 
and  across  situations  and  time.  The  upshot  of  this  diverse  environment  is  that  community  members  progress  in  their 
epistemic  understanding,  perhaps  moving  from  black-and-white  views  of  knowledge  toward  more  sophisticated  views  of 
how  we  come  to  know  things. 

Community  members  who  are  responsible  and  skilled  at  diagnosing  and  addressing  their  learning  needs . Here  is  both  a 
benefit  and  a challenge  to  DLCs.  When  control  is  distributed  throughout  the  group,  more  demands  are  placed  on 
individual  members.  Because  teacher  is  no  longer  doing  the  hard  work  of  deciding  on  goals,  methods,  and  new 
knowledge,  community  members  must  meet  the  challenge  of  assuming  these  roles.  Metacognitive  knowledge — knowing 
how  to  monitor  one’s  learning  and  how  to  address  ill-defined  problems — becomes  an  essential  part  of  the  community, 
which  hopefully  can  also  be  shared  throughout  the  group.  A systemic  analysis  may  conclude  that  a given  group  cannot 
become  a DLC  because  of  deficiencies  in  this  area.  On  the  other  hand,  a group  may  progress  incrementally  in  these 
skills  and  move  steadily  toward  more  self-  (or  community-)  directed  learning. 

Negatives 

Short-term  inefficiencies . Just  as  a Vermont  town  meeting  can  be  more  laborious  and  inefficient  than  a professionally 
managed  city’s  well-defined  processes,  so  DLCs  can  be  more  inefficient  and  indirect  than  controlled  instruction.  A well- 
packaged  instructional  program  may  be  able  to  teach  a fixed  set  of  rules  more  efficiently.  If  such  a product  were  reliably 
available,  then  a DLC  would  be  wise  to  recommend  its  use.  In  the  absence  of  such  structure,  however,  DLCs  may  tend 
to  “muddle  through”  (cf.  Bateson,  1972,  pp.  3-8)  with  its  share  of  redundancies,  inefficiencies,  lack  of  focus,  and  lengthy 
processes. 

In  the  long  term,  however,  DLCs  may  be  an  efficient  route  toward  learning.  A cow  does  not  look  for  the  shortest 
route  up  a hill,  but  rather  keeps  its  head  down  and  looks  for  a steady  way  up  without  noticeable  climbing.  The  result  is 
the  most  efficient  use  of  its  energy  (Allen,  1996).  DLCs  will  tend  to  meander.  But  like  the  cow  up  the  hill,  the  shortest 
path  may  not  always  be  the  wisest  path,  and  certainly  not  the  most  efficient  path. 


I4Note  our  attribution  of  intentionality  to  the  DLC  (it  “wants”  this  or  “seeks”  that).  This  issue  is  somewhat  controversial; 
for  example,  Bateson  (1972)  avoids  such  language  because  it  leads  to  a category  error — thinking  that  systems  have  a mind  in 
the  same  way  humans  do.  Part  of  our  message,  however,  is  that  complex  systems  do  seem  to  have  minds  of  their  own;  that 
is,  they  come  to  behave  as  though  they  had  intentions.  Whether  this  crosses  a boundary  of  appropriate  discourse,  and  moves 
into  an  anthropomorphic  error,  we  will  speak  of  systems  as  having  intentionality  because  it  is  a useful  shorthand  for 
understanding  complex  systemic  behavior. 
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Lack  of  control.  DLC’s  decentralized  control  can  be  a handicap.  The  leadership  and  vision  of  a charismatic  leader  can 
marshal  community  resources  and  stimulate  purposive  action.  At  times,  the  unwieldiness  or  fuzziness  of  a DLC  can 
frustrate  those  who  would  like  to  see  decisive  action  or  clear  direction  for  learning. 

Lack  of  predictability.  DLCs  can  frustrate  the  intentions  of  the  best  designers.  A constant  among  initiators  of  DLCs 
is  the  reported  surprise  at  the  direction  the  group  takes.  DLCs  seem  to  have  a mind  of  their  own,  and  where  they  end  up 
is  not  where  they  start.  This  is  true  of  distance-learning  groups,  workgroups,  and  learning  communities  within 
classrooms.  Often  the  surprises  are  pleasant,  but  the  evolving  nature  of  the  group  can  be  difficult  for  people  trying  to 
plan  for  the  future. 

In  one  sense,  DLCs  can  be  thought  of  as  being  learner-centered.  Community  members  must  take  more  responsibility 
for  their  own  learning  than  in  most  designed  instructional  systems.  In  another  sense,  however,  the  centeredness  is 
found  in  the  community  rather  than  the  individual  learner.  Ideally,  community  members  lacking  metacognitive  skills 
may  participate  and  receive  support  from  the  group.  The  group  often  dictates  the  learning  agenda— or  at  least  engages 
individuals  in  dialogue  concerning  that  agenda.  The  thought  of  individuals  isolated,  setting  individual  goals,  pursuing 
those  learning  goals  individually— This  is  contrary  to  our  conception  of  a dynamic  learning  community.  So  we  tend  to 
think  of  DLCs  not  simply  as  tools  for  self-directed  learning,  but  as  supportive  communities  wherein  a variety  of  learning 
goals  may  be  pursued,  some  individual  and  some  shared  throughout  the  membership. 

Three  Scenarios 

In  this  section,  we  present  three  scenarios  we  have  observed,  where  dynamic  learning  communities  (or  DLCs)  are 
beginning  to  take  root. 

Workplace  learning.  Martin  works  with  a group  of  engineers  charged  with  developing  new  products  and  specifications 
for  supporting  those  new  products.  Martin  finds  himself  working  on  products  whose  standards  have  not  yet  been 
finalized,  which  in  any  case  will  be  replaced  within  18  months  by  another  standard  or  a newer,  more  capable  product. 

When  a standard  doesn’t  exist,  where  does  an  engineer  go  to  get  answers?  Off-the-shelf  training  is  no  help  except  for 
generic  skills.  Customized  training  products  such  as  computer-based  training  take  too  long  too  develop  and  would  not  be 
cost-effective  for  the  specialized  needs  of  the  small  engineering  workgroup.  Hiring  a consultant/trainer  to  come  from  the 
university  and  give  lectures  is  a possibility,  but  often  the  expertise  is  not  available,  and  when  it  is,  the  costs  of  pulling 
people  away  from  their  work  in  a high-pressure  environment  can  be  enormous.  Even  traditional  performance  support 
systems— electronic  or  otherwise — exact  a toll  in  time  and  effort.  These  systems  must  be  designed,  and  therein  lies  the 
problem:  nobody  knows  enough  to  design  them,  and  if  they  did,  they  would  be  too  busy  putting  out  immediate  fires  to 
take  the  time.  In  short,  expertise  is  scarce  and  doesn't  exist  in  any  form  specifically  designed  for  instruction  or  support. 

As  an  alternative  to  traditional  training  and  performance-support  solutions,  Martin  has  been  promoting  the  concept  of 
shared  problem  solving  and  archiving  of  solutions.  When  an  engineer  needs  help,  she  asks  for  help  among  the 
workgroup.  If  someone  has  an  answer,  the  solution  is  shared  publicly  and  archived  for  future  reference.  Getting 
engineers  to  think  in  terms  of  mutual,  collaborative  support  is  a challenge,  but  given  the  pace  of  change  and  the  demands 
for  expertise,  they  really  have  no  choice.  They  must  learn  to  share  expertise,  or  they  will  not  survive  in  their 
competitive  environment.  This  general  approach  of  mutual  support  for  learning  is  further  discussed  in  later  sections  of 
the  paper. 

Academic  culture.  What  is  an  academic  program?  Is  a masters  program  the  sum  of  courses  required  of  students  for  the 
degree,  or  is  it  something  more?  How  does  an  academic  unit’s  local  culture  serve  to  encourage  learning — within  classes, 
on  collaborative  projects,  or  among  individual  students  and  faculty?  Brent  has  been  reflecting  on  ways  that  students, 
faculty,  staff,  friends,  and  alumni  all  work  together  to  foster  learning  and  professional  growth.  The  communications 
infrastructure  provided  by  e-mail  and  the  World-Wide  Web  can  serve  to  facilitate  higher  levels  of  connectivity  and 
participation  and  new  learning. 

In  the  case  of  higher  education,  faculty  members  benefit  as  much  as  students  from  the  interaction  and  sharing  of 
expertise.  Because  faculty  members  do  not  typically  return  to  school  for  more  degrees,  they  rely  on  professional 
interactions — including  stimulus  from  students — as  a key  resource  for  new  learning. 

For  the  last  couple  of  years,  Brent  and  colleagues  have  been  exploring  ways  to  strengthen  the  collaborative  sharing  and 
out-of-class  learning  that  naturally  occurs  in  and  around  the  academic  program  (see  Wilson,  Ryder,  McCahan,  & Sherry, 
1°96  for  a report  of  their  work).  Students  with  resources  of  their  own  become  less  dependent  on  professors  and  courses 
as  sources  of  expertise,  and  move  toward  a wider  variety  of  learning  activities  and  interactions.  Over  time,  these  informal 
interactions  come  to  constitute  a learning  community,  and  become  as  important  to  the  education  of  participants  as  formal 
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Internet  discussion  groups . Globally,  a swell  of  informal  or  distributed  learning  initiatives  have  taken  shape,  using  the 
Internet  as  its  medium.  Indeed,  the  Internet  serves  as  a sort  of  “petri  dish”  wherein  a variety  of  informal  cultures  have 
begun  sprouting  up.  Many  of  these  learning  initiatives  are  independent  of  traditional  instruction.  Participants  in  a 
listserv  such  as  IT  Forum,  for  example,  may  engage  in  high-level  discussions  concerning  technology  in  education,  yet 
their  participation  may  not  be  reflected  in  course  credit  and  may  not  be  governed  by  a teacher  or  instructional  designer 
(Rieber,  1996).  People  may  especially  benefit  from  participating  on  global  forums  when  expertise  is  rare  within  their 
local  environment.  In  a way,  the  monopoly  of  expertise  is  being  shaken  loose  from  the  universities,  big  businesses, 
large  cities,  and  developed  countries,  and  is  being  distributed  throughout  the  world  via  the  Internet.  The  opportunities 
afforded  by  new  communications  technologies  will  eventually  have  profound  implications  for  how  we  think  about 
learning  and  instruction. 


The  DLC  Learning  Process:  The  Dialogical  Case 
Learning  can  happen  in  a variety  of  ways  within  a DLC;  however,  a pattern  of  mutually  support  will  tend  to 
emerge,  outlined  in  Table  2 below.  Each  step  is  described  in  turn. 

1 . Articulate  the  learning  need.  This  becomes  the  learning  “problem”  or  goal. 

2.  Seek  help  in  a group  forum. 

3.  Engage  in  a help  consultation. 

4.  Assess  learning. 

5.  Share  the  solution  with  the  group.  Restate  the  problem  and  solution  if  necessary. 

6.  Archive  the  interaction  or  the  restated  solution  for  future  reference. 

7.  Repeat  this  process,  of  any  part,  as  necessary  to  support  learning. 

Table  2.  A dialogical  approach  to  learning  within  DLCs. 

Articulate  the  learning  need . A community  member  becomes  aware  that  she  or  he  lacks  some  skill  or  knowledge.  The 
learning  need  may  not  be  fully  analyzed;  that  is,  how  the  needed  knowledge  relates  to  a particular  problem  or  performance 
need  not  be  fully  specified  at  the  outset.  The  need  to  know  becomes  a “problem”  or  learning  goal  for  the  individual. 

Seek  help  in  a group  forum.  The  community  member  then  seeks  help,  often  in  a public  forum,  such  as  a distribution 
list  or  listserv  maintained  by  the  DLC. 

Engage  in  a help  consultation . Another  community  member  helps  or  consults  with  the  first  member.  Help 
consultations  may  draw  on  a variety  of  resources: 

— human  resources; 

— archived  interactions; 

— FAQs; 

— information  search  tools; 

— performance  supports; 

— instruction. 

Community  members  may  discuss  the  issue  at  length,  publicly — Or  the  help  consultation  may  be  simple,  direct,  and 
private. 

Assess  learning.  Community  members  have  a variety  of  tools  to  use  in  testing  out  new  knowledge  or  skill.  If  the 
help  consultation  provides  incomplete  information,  the  community  member  may  succeed  in  filling  in  the  missing 
information.  A recommended  procedure  can  be  tried  out;  if  it  fails  and  the  problem  can’t  be  solved,  the  person  goes  back 
and  reports  the  problem  and  repeats  the  interactive  process.  If  new  knowledge  is  offered,  it  is  tested  against  prior 
knowledge  and  understanding.  New  knowledge  can  also  be  compared  across  members  of  the  group.  Typically,  a 
combination  of  self-assessment  and  consensual  agreement  is  the  norm. 

Share  the  solution  with  the  group.  After  new  knowledge  or  skill  is  tried  out  and  confirmed,  the  solution  is  shared 
with  the  group.  A restatement  of  the  original  problem  and  its  solution  may  be  helpful,  especially  for  future  reference. 

Archive  for  future  reference.  Ideally,  every  DLC  interaction  should  be  archived  for  future  retrieval.  If  an  automatic 
archiving  system  is  not  in  place,  then  each  solved  problem  or  significant  interaction  should  be  stored  in  a public  location 
for  future  access  by  any  member  of  the  group. 

Repeat  as  necessary.  Although  listed  in  steps,  the  general  process  is  flexible.  Any  step  or  set  of  steps  may  be  repeated 
in  the  process  of  generating  solutions  to  learning  problems. 
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The  process  above  typifies  interactions  where  a specific  learning  need  is  identified.  A common  alternative  is  the  kind 
of  informal  discussion  characteristics  of  many  listservs  and  discussion  groups.  Frequently,  knowledge  sharing  is  not 
problem-driven  but  rather  conversation-driven.  In  those  cases,  the  learning  activity  is  less  strategic  and  less  defined,  but 
learning  is  nonetheless  supported  by  mutual  sharing  and  concern  around  a topic  of  interest. 

Managing  Expertise  in  the  DLC 

One  challenge  DLCs  have  is  keeping  both  experts  and  novices  happy  interacting  within  the  same  group.  Novices  have 
a tendency  to  ask  inappropriate  questions  and  to  not  use  the  full  resources  available  to  them.  Experts  can  feel 
overworked,  exploited,  or  unchallenged — feeling  that  they  aren’t  learning  anything.  How  does  a learning  community  deal 
with  the  varying  needs  of  its  members?  Before  we  specifically  address  this  question,  a few  general  comments  are  in  order. 

Expertise  is  relative  and  multi-faceted.  Expert  and  novice  roles  fluctuate  within  the  community.  Brent  is  an  expert  at 
one  thing,  Martin  at  another.  Some  members  may  be  expert  at  group  cohesion,  serving  a critical  support  role  in  keeping 
the  community  together.  Others  may  participate  irregularly,  but  have  important  input  at  certain  junctions.  Certainly 
labels  such  as  “expert”  or  “novice”  do  not  capture  the  richness  of  the  expertise  that  is  distributed  throughout  a learning 
community. 

There  must  be  a legitimate  self-interest  to  sustain  individual  participation.  At  the  individual  level,  community 
members  must  each  feel  a personal  return  on  their  investment  in  the  group.  In  this  sense,  a perceived  self-interest  must 
accompany  continuing  involvement  with  the  community.  There  are,  however,  a number  of  grounds  for  self-interest. 
Some  people  get  tremendous  satisfaction  out  of  helping  others.  Some  people  like  to  think  of  themselves  as  being  a 
source  of  expertise  to  others.  Some  people  have  a need  to  feel  connected  to  groups  of  people,  even  when  significant  new 
learning  happens  only  irregularly.  Some  people  develop  a sense  of  loyalty  and  constancy,  committing  time  and  resources 
to  a community  because  it  helps  them  find  a “home.”  All  of  these  may  be  reasons  why  a person  with  considerable 
expertise  may  choose  to  participate  in  a diverse  learning  community. 

Martin  has  an  example  from  engineering.  An  Internet  discussion  list  exists  around  new  standards  for  the  next 
generation  of  the  SCSI  interface.  List  members  have  a variety  of  motives  for  participating.  A sales  and  marketing 
person  may  feel  like  a novice,  with  a need  to  upgrade  his  understanding  of  technical  details.  An  experienced  engineer  may 
participate  because  she  feels  a need  to  maintain  a link  to  “people  out  there”  and  understand  people  s reactions  to  proposals 
and  possibilities.  This  knowledge  will  then  help  her  in  establishing  better  standards.  The  list  thrives  because  people 
depend  upon  each  other  and  need  the  different  kinds  of  expertise  available  among  the  group. 

We  return  again  to  the  different  needs  of  novices  and  experts  within  learning  communities.  Novices  often  feel 
intimidated,  or  may  tend  to  under-utilize  available  resources,  while  experts’  time  is  often  overtaxed  with  questions  and 
service  demands,  leaving  little  time  for  the  renewal  of  their  own  expertise.  We  offer  several  possible  strategies  that 
DLCs  might  choose  to  adopt  in  addressing  the  general  problem,  summarized  in  Table  3 below. 


For  Everyone: 

• Varied  levels  of  discussion  and  activity  (e.g.,  beginners  vs.  advanced) 

• A process  that  inducts  beginners  and  moves  them  through  increasing  levels  of  expertise 

• Moderated  lists 
For  the  Expert: 

• Public  and  private  accounts 

• Allowing  anonymous  posts 

• Private  groups 

• Temporary  consultations,  participation 
For  the  Novice: 

• User-friendly  search  and  navigation  tools 

• Private  consultations  and  re-directions 

• Paid  advisors/help  specialists  (e.g.,  AOL  guides) 

• Standard  problem-solving  protocols  (e.g.,  "have  you  read  our  FAQ?") 

Table  3.  Possible  strategies  for  managing  expertise  within  dynamic  learning  communities. 

The  strategies  suggested  in  Table  3 are  possibilities  only.  Several  would  curtail  or  constrain  free  interaction  within  the 
group,  thus  inhibiting  the  “dynamic”  nature  of  the  learning  community.  Care  should  be  taken  not  to  fix  a problem  that 
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doesn’t  exist.  Specific  policies  can  best  be  determined  by  consensus  among  the  local  community,  and  evolve  over  time 
as  needs  change. 


ID  VERSUS  DLCs? 

Both  instructional  design  (ID)  and  DLCs  can  lead  to  learning.  However,  other  things  being  equal,  we  believe  that  open 
systems  are  preferred  because  they  address  more  fundamental  learning  outcomes,  e.g.,  self-directed  inquiry,  learning-how- 
to-leam,  metacognition,  etc.,  and  are  more  closely  situated  within  a natural  performance  environment.  The  following 
table  presents  an  outline  of  variables  to  consider  when  choosing  between  DLCs  and  designed  instructional  systems. 


Use  Instructional  Design  if  these  conditions  apply: 

• stable  content  over  time; 

• well-defined  content; 

• best  for  algorithms  and  rules; 

• heavy  representation  demands; 

• mastery  of  discrete  knowledge  is  valued. 

Try  DLCs  if  these  conditions  apply: 

• volatile,  changing,  or  new  content; 

• ill-defined  content; 

• best  for  complex  problems  and  content; 

• heavy  literacy  and  metacognitive  demands; 

• community-directed,  situated  support  for  learning  is  valued. 

The  following  are  relevant  (but  NOT  determining)  factors: 

• level  of  expertise; 

• criticality  of  content. 

Table  4.  Factors  to  consider  when  deciding  between  designed  instruction  and  dynamic  learning  communities. 

Note  that  the  criticality  of  the  learning  or  the  advancedness  of  the  skills  are  not  determining  factors  when  deciding 
between  designed  instruction  and  DLCs.  That  is,  DLCs  can  be  effectively  used  at  varying  levels  of  expertise,  and  with 
critically  important  content.  Where  certification  of  expertise  is  necessary,  however,  individuals  should  demonstrate  their 
expertise  using  accepted  assessment  methods,  regardless  of  the  method  of  learning  support. 

In  general,  the  more  stable,  defined,  and  discrete  the  content,  the  more  sense  to  design  instruction  to  meet  the  learning 
need.  Contrariwise,  the  more  volatile,  ill-defined,  and  complex  the  learning  needs,  the  more  sense  to  try  dynamic 
learning  communities  as  a support  strategy. 

In  the  end,  the  decision  between  designed  instruction  and  DLCs  is  partly  one  of  utility  and  partly  one  of  value.  That 
is,  given  the  same  ends,  DLCs  may  prove  more  or  less  effective  in  accomplishing  learning.  Seen  in  this  way,  the  issue 
is  one  of  utility.  Taken  the  next  level,  however,  designed  instruction  cannot  be  said  to  accomplish  the  same  ends  as 
DLCs,  and  vice  versa.  At  this  point,  the  decision  necessarily  rests  on  the  question:  What  learning  ends  do  we  really 
value?  Members  of  dynamic  learning  communities  will  come  out  of  their  experience  with  different  skills,  perspectives, 
and  appreciations  than  graduates  of  an  instructional  program.  These  differences  must  be  respected  and  considered  when 
choosing  between  approaches  to  learning  support — in  addition  to  the  utilitarian  considerations  mentioned  above. 

Costs.  Traditionally  designed  instruction  requires  relatively  low  investment  to  establish  the  culture  because  most 
school  and  training  cultures  are  already  in  place.  However,  heavy  investment  is  required  for  each  new  instructional 
product  and  for  continuing  deliveiy  of  instruction.  DLCs,  on  the  other  hand,  require  heavy  front-end  investment  to 
establish  a culture  and  support  new  users.  Once  users  become  experienced,  the  costs  can  be  expected  to  be  lower.  At  this 
point,  we  really  don’t  know  enough  about  cost  comparisons  to  make  definitive  judgments.  More  data  on  this  subject 
will  be  helpful  as  DLCs  become  more  heavily  used  and  studied. 

Control  of  DLCs 

As  we  made  emphasized  above,  because  DLCs  are  open  systems,  control  is  distributed  throughout  the  community. 

This  can  be  both  a strength  and  a weakness.  Examples  of  problems  that  can  arise  include: 
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— pornography  access  in  a middle  school; 

— bomb-assembly  instructions  two  or  three  links  away  from  your  homepage;^ 

— competitor  "air  space"  on  company-sponsored  net; 

— flaming  and  dissent  on  discussion  groups; 

— addictive  and  inappropriate  behaviors; 

— mismatch  between  DLC  learning  and  externally-defined  curriculum  objectives. 

Deciding  how  to  deal  with  these  kinds  of  problems  brings  us  back  to  the  core  differences  between  open  learning 
systems  and  closed  instructional  systems.  We  may  choose  to  respond  to  problems  by  seeking  to  limit  the  openness  of 
resources  available  to  a DLC.  Possible  methods  for  exerting  some  measure  of  control  over  DLCs  include: 

— list  moderation; 

— control  over  membership; 

— externally  mandated  rules  and  conventions; 

— imposed  problems  or  learning  activities; 

— imposed  assessment  standards. 

Such  attempts  are  controlling  DLCs  constitute  a compromise  of  their  open  nature.  Thus  the  learning  community 
becomes  something  of  a hybrid  between  a DLC  and  a designed  instructional  system.  Such  compromises  may  be 
necessary  in  schools  or  other  real-life  settings,  but  they  should  be  implemented  very  carefully,  since  interventions  can 
have  unpredictable  effects  on  group  functioning.  This  again  is  another  area  that  we  know  very  little  about,  in  need  of 
further  research. 


Conclusion 

Heretofore,  instructional  designers  have  thought  they  were  in  the  business  of  designing  instructional  systems  to  meet 
prespecified  learning  objectives.  But  first  the  constructivist  movement — and  now  communications  technologies 
themstlves — seem  to  be  threatening  this  conception  as  the  sole  way  to  support  learning.  People  are  learning  without 
help  from  designed  instruction!  In  many  settings,  in  fact,  “natural”  learning  is  more  prevalent  than  “designed”  learning 
(Resnick,  1987).  We  believe  that  the  situation  requires  a reexaminination  of  our  core  roles.  Are  we  in  the  business  of 
designing  instruction  or  are  we  in  the  business  of  supporting  valuable  learning,  wherever  it  may  happen?  The  answer  to 
this  question  will  result  in  either  a narrow  or  broad  interpretation  of  our  role  and  its  relationship  to  non-instructional 
forms  of  learning. 

Our  own  belief  is  that  dynamic  learning  communities  are  proper  objects  of  study.  We  should  seek  to  understand  how 
such  communities  function,  how  they  grow,  how  they  can  be  nurtured,  and  how  they  can  be  replicated  across  diverse 
settings.  But  nurturing  is  different  than  designing.  We  must  respect  the  integrity  of  the  community.  In  time,  we  may 
come  to  think  of  ourselves  more  as  “learning  technologists”  than  as  “instructional  technologists”,  and  “learning  support 
specialists”  more  than  “instructional  designers.”  But  these  are  issues  best  addressed  at  length  in  a separate  paper. 

In  conclusion,  the  development  of  new  communications,  storage,  and  representation  technologies  constitute  a 
watershed  in  the  history  of  open  learning  environments,  making  DLCs  more  feasible  than  they  have  been  in  the  past. 
This  is  a situation  where  the  technology  allows  a concept  to  take  shape,  and  the  interplay  between  technology  and  theory 
will  likely  continue  in  the  years  to  come. 

The  decision  rule  concerning  DLC  versus  ID  may  slide  toward  DLCs  as  we  learn  more  about  what  works.  In 
particular,  we  need  to  better  understand  how  established  instructional  systems  (e.g.,  school  classrooms)  can  migrate 
toward  greater  openness,  eventually  resulting  in  a displacement  of  instruction  for  a community  model.  A transition 
model  that  outline  this  growth  trajectory  would  be  a most  welcome  research  agenda  in  the  coming  years. 

In  the  meantime,  we  will  continue  studying  how  dynamic  learning  communities  take  shape,  how  they  self-organize, 
and  how  they  support  learning.  Documenting  cases  empirically  is  an  important  part  of  that  agenda,  and  will  help  to 
clarify  several  issues  merely  touched  on  in  this  paper. 


15This  example  comes  from  our  own  experience  maintaining  IT  Connections  (http://www.cudenver.edu/-mryder/itcon.html). 
In  response  to  a student’s  complaint,  we  could  only  shrug  our  shoulders  and  mumble  something  about  the  wonders  of  an  open 
system! 
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Abstract 

Despite  enormous  growth  of  the  Internet  and  its  proliferation  of  tools,  resources,  and  on-line  communities,  the 
connection  between  local  learning  environments  and  virtual  learning  environments  remains  tenuous.  This  paper 
examines  this  relationship,  based  upon  a case  study  of  an  academic  unit  at  the  University  of  Colorado  at  Denver. 
Drawing  on  interviews,  written  surveys,  and  exchanges  with  students,  we  offer  an  analysis  of  factors  that  are 
critical  to  a user’s  choice  to  participate  in  the  Internet.  The  paper  concludes  with  recommended  strategies  for 
introducing  students  and  faculty  to  networked  learning  environments. 

The  Internet  has  sustained  phenomenal  growth  since  the  beginning  of  the  decade.  Doubling  in  size  each  year  for  the 
past  several  years,  the  Internet  now  connects  35  million  users  across  the  globe.  Three  million  new  accounts  are  added 
each  month.  Higher  education  accounted  for  most  of  the  growth  in  the  early  nineties,  but  today,  commercial  usage 
accounts  for  65%  of  network  traffic.  The  public  schools  are  also  wired.  The  U.  S.  Department  of  Education  estimates 
that  over  50  percent  of  American  Schools  will  be  attached  to  the  Internet  by  the  end  of  1996.  Despite  this  astounding 
growth,  the  majority  of  Americans  at  this  moment  remain  unwired,  anticipating  nothing  extraordinary  about  the  age  of 
information. 

In  spite  of  the  Net’s  growth,  most  people  remain  on  the  sidelines,  hearing  media  reports  but  staying  a cautious  distance 
away  from  time-consuming  forays  into  the  medium.  A college  campus  is  a microcosm  of  the  larger  community.  Some 
students  spend  hours  in  exploration  of  virtual  worlds,  some  engage  in  public  forums  on  specific  issues,  some  utilize  the 
Net  for  its  research  capabilities,  while  many  adopt  the  convenience  of  electronic  mail.  A substantial  body  of  faculty  and 
students  remains  largely  unaware  of  the  full  array  of  resources  available  to  them. 

Internet  as  a Technical  Innovation 

One  way  to  look  at  the  growth  of  the  Internet  is  as  a new  technology  being  adopted.  Rogers  ( 1995)  has  developed  an 
influential  model  for  the  adoption  and  diffusion  of  innovations  within  organizations;  theorists  continue  to  emphasize  that 
today's  organizations  must  be  willing  and  ready  to  undergo  continual  change  (e.g.,  Senge,  1990).  Seen  in  this  way,  a 
department  head  or  office  manager  might  develop  a plan  for  implementing  the  technology,  similar  to  the  installation  of  a 
new  computer  or  LAN.  Consistent  with  this  adoption/diffusion  approach,  Internet  resources  can  be  seen  as  "mere  tools” 
to  be  utilized  by  existing  organizations  and  individuals.  Through  the  Internet,  an  individual  may  solve  a problem,  find  an 
answer  to  a question,  or  communicate  with  another  professional  on  a project. 

The  Internet  differs,  however,  from  many  technological  innovations  in  that  it  seems  to  be  largely  driven  from  the 
bottom— from  the  academic  community  and  from  individuals  and  small  groups  of  grass-roots  enthusiasts.  Organizations 
find  themselves  responding  to  an  unplanned  cultural  shift,  rather  than  implementing  a new  technology  from  the  top 
down.  Moreover,  academic  units  in  higher  education  often  lack  the  cohesion  of  a traditional  business  office.  Professors 
are  highly  autonomous  workers  who  spend  more  time  interacting  with  students  than  with  colleagues.  Developing  a 
consensus  for  technological  change  can  be  a serious  challenge  in  such  settings. 

Internet  as  a Second  Culture. 

Another  way  of  viewing  the  Internet  is  as  a second  culture  or  community  (Rheingold,  1993).  The  Internet  exhibits  all 
the  key  elements  of  a culture,  including  language,  symbols,  rituals,  status,  and  other  meaning-conveying  forms 
(December,  1993;  North,  1995).  Individuals  may  "enter"  this  environment,  become  initiated  into  various  sub-groups,  and 
interact  with  other  community  members.  At  the  group  level,  an  entire  company  or  department  may  introduce  its 
members  to  the  Internet  culture. 

This  broader  conception  underscores  the  depth  of  change  that  the  Internet  poses  to  existing  organizations.  One  cannot 
expect  to  simply  "tack  on”  the  Internet  as  a cosmetic  addition  to  an  existing  structure;  as  a new  and  competing  culture, 
the  Internet  is  bound  to  threaten  existing  conventions  and  cultural  practices. 

The  gap  between  the  preexisting  local  community  (typically  an  academic  or  business  unit)  and  the  virtual  Internet 
community  is  worthy  of  study.  How  and  why  do  individuals  move  toward  the  Internet  culture?  How  does  an  entire  group 
of  people  make  the  move?  What  factors  are  at  play,  and  what  can  be  done  to  facilitate  the  change? 

This  paper  draws  on  our  own  efforts  to  bring  our  local  academic  culture  in  line  with  the  Internet  culture.  Over  the  last 
couple  of  years,  we  have  been  using  the  Internet  within  our  academic  unit,  the  program  in  Information  and  Learning 


Technologies  (ELT)  at  the  University  of  Colorado  at  Denver  (UCD).16  We  have  been  intrigued  to  observe  the  response 
of  faculty  and  students  as  they  gravitate  toward  the  Net.  An  expected  normal  distribution  occurs,  ranging  from  anxiety, 
frustration,  and  resistance  to  utilitarian  accommodation  to  excitement  and  immersion.  In  some,  these  differing  responses 
reflect  personal  stages  of  growth  over  time.  We  hope  the  local  academic  community  can  overcome  initial  fears  and 
frustrations  and  move  toward  mature  utilization  of  Internet  resources  much  as  they  might  acquire  a second  language 
through  a semester  abroad  program.  Our  hopes,  however,  have  not  yet  become  reality. 

Our  purpose  in  this  paper  is: 

1 . to  reflect  on  the  general  problem  of  the  local  and  Internet  cultures; 

2.  to  report  on  our  efforts  to  support  the  integration  of  the  two  cultures;  and 

3.  to  point  to  areas  of  needed  research  and  to  offer  recommendations  to  designers  of  learning  environments  for  the 
successful  integration  of  the  Internet  into  existing  learning  cultures. 

Beginning  Efforts 

Martin  Ryder  wrote  a paper  in  1994  which  articulated  his  vision  of  the  Internet  as  a learning  tool  (Ryder,  1994).  To 
test  out  his  developing  ideas,  he  concurrently  developed  a homepage  to  represent  the  local  department’s  interests  and  self- 

concept.  Instructional  Technology  (IT)  Connections17  is  an  all-purpose  hypertext-based  help  page  for  beginning  and 
experienced  users  of  UCD  computers.  Located  outside  the  UCD  network's  firewall,  on  a gopher  server  named  Oasis,  IT 
Connections  links  students  with  a variety  of  Internet  databases  and  tools  relating  to  instructional  technology  and 
cognitive  science.  IT  Connections  has  been  accessible  on  the  World-Wide  Web  since  the  spring  of  1994,  when  it 
appeared  in  the  Educational  Technology  section  of  CERN's  virtual  library,  Stanford's  Connections  to  Cognitive  and 
Psychological  Sciences,  the  Curriculum  and  Instruction  section  of  EINet  Galaxy,  and  the  Education  section  of  John 
December's  list. 

The  Online  Helpdesk  is  a key  component  of  IT  Connections  designed  to  provide  documentation,  tools  and  other 
supports  for  new  users.  It  includes  direct  access  to  gopher  resources,  Usenet  and  world-wide  web  utilities,  allowing  the 
user  to  access  them  in  a seamless  environment.  While  the  Helpdesk  was  specifically  designed  for  new  users,  experienced 
users  maintain  active  links  to  the  resource  for  its  exhaustive  FAQ  lists,  interest  group  databases,  and  practical  search 
tools.  For  more  complete  descriptions  of  IT  Connections,  the  Online  Helpdesk,  and  other  performance  supports,  see 
Sherry  et  al.  (1995). 

Hiding  the  Complexity 

Powerful  resources  have  been  available  to  users  within  our  local  department  from  the  start,  including  Usenet,  gopher, 
ftp,  and  telnet.  But  these  tools  required  the  burden  of  rudimentary  Unix  skills  which  effectively  kept  the  resources  out  of 
reach  of  most  new  users.  Not  surprisingly,  they  were  minimally  used. 

Our  tactical  approach  was  to  eliminate  any  direct  reference  to  such  tools  whenever  possible.  Martin,  who  works  as  a 
software  engineer  in  his  daytime  job,  designed  command  scripts  which  enable  users  to  perform  complex  Unix  operations 
with  simple  commands.  The  command,  ’'connect”,  ftu  example,  invokes  a script  that  brings  up  IT  Connections  with  the 
Lynx  browser,  a tool  designed  at  the  University  of  Kansas  specifically  with  new  users  in  mind.  For  local  users,  IT 
Connections  serves  as  their  window  to  Internet  services  in  education.  The  single  command  "connect"  allows  the  user 
direct  access  to  IT  Connections  without  requiring  technical  knowledge  of  the  underlying  details.  A similar  command, 
"aid”,  invokes  Lynx  to  bring  up  the  Online  Helpdesk. 

Another  tool,  "makepage",  was  developed  to  allow  new  users  to  immerse  themselves  in  online  authoring  without 
passing  through  the  initiation  rites  of  Unix.  The  single  command  "makepage"  creates  for  the  user  a public  directory  with 
an  index  template.  It  sets  all  the  appropriate  security  permissions  to  allow  public  browsing  of  the  new  directory.  The 
new  user  is  not  encumbered  with  these  distracting  cognitive  tasks.  After  invoking  the  command,  users  can  immediately 
begin  editing  an  html  template,  customizing  it  to  their  own  specifications  as  they  begin  to  mold  a personal  homepage 
from  a generic  model. 


16ILT  programs  are  housed  within  the  Division  of  Technology  and  Special  Services,  which  is  within  the  Graduate  School  of 
Education. 

,7The  URL  to  IT  Connections  is:  gopher://www.cudenver.edu/-mryder/itcon.html 
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Such  conveniences  do  not  come  without  a price.  Users  who  wish  to  understand  the  technical  details  of  Unix  and  html 
are  not  directly  rewarded  with  this  approach.  But  the  templates  and  shells  provided  by  the  tools  have  opened  the  door  for 
many  users  whose  interests  are  focused  beyond  the  technology. 

The  Internet  Task  Force 

In  the  process  of  developing  these  support  tools,  Martin  learned  something  interesting:  He  could  barely  get  the 
attention  of  anyone  around,  student  or  faculty.  This  was  especially  ironic,  since  the  department  was  ostensibly 
committed  to  educational  technologies  as  its  professional  focus.  Professors  in  the  program  had  been  teaching 
telecommunications  classes  and  using  e-mail  for  years.  Why  didn't  anybody  share  Martin's  excitement?  Why  did  faculty 
members  take  weeks  to  even  take  a look  at  the  Web  resource,  IT  Connections?  Martin  expressed  a sense  of 
disequilibrium  and  confusion  as  he  contrasted  the  supportive  responses  of  remote  patrons  with  the  general  disinterest 
shown  locally. 

Undeterred  by  the  resistance  shown  to  the  new  tools,  Martin  joined  faculty  member  Brent  Wilson  and  a team  of 
students  to  study  the  issue.  The  Internet  Task  Force  was  created  in  the  fall  of  1994,  with  the  following  broad  objectives:  - 

• to  support  the  department  in  its  move  toward  the  Internet  culture  through  a variety  of  online  and  offline  tools, 
support,  training,  and  policy  initiatives; 

• to  develop  ways  that  e-mail  and  the  Internet  could  be  used  as  knowledge-building  tools  within  graduate  classes  and 
seminars; 

• to  reflect  on  and  conduct  research  on  users'  needs,  support  tools,  adoption  processes,  cultural  change,  and 
collaborative  learning  communities. 

Our  overall  goals  for  student  (and  faculty)  online  access  involve  the  ability  to  engage  actively  within  local  and  virtual 
communities  toward  the  social  construction  of  public  knowledge.  Specifically,  we  want  students  and  faculty  in 
Information  and  Learning  Technologies  to  utilize  online  resources  in  order  to: 

• Engage  in  public  dialog  over  issues  of  scholarly  and  professional  import. 

• Exploit  the  various  resources  that  comprise  a public  knowledge  base. 

• Contribute  information  of  value  to  the  public  knowledge  base. 

• Provide  consulting  and  collaborative  services  to  communities  of  learners,  scholars  and  practitioners. 

The  group  has  met  regularly  since  August  1994,  and  has  created  a number  of  performance  supports  and  research 
products  related  io  educational  uses  of  the  Internet.  Sherry  et  al.  (1995)  provides  a more  complete  report  of  the  Task 
Force  activities  and  accomplishments. 


User  Survey 

In  spite  of  our  enthusiasm  for  the  potential  of  Internet  resources,  we  were  sensitive  to  the  very  real  obstacles  facing 
students  and  faculty  alike.  In  particular,  we  were  wary  of  mandating  policy  and  enforcing  changes  in  the  culture  without 
careful  assessment  of  people's  perceptions  and  needs  (Sarason,  1988).  We  designed  a written  questionnaire  based  on 
Rossett’s  (1991)  needs  assessment  model,  which  was  intended  to  survey  the  level  of  Internet  training  and  support  needed 
by  our  students,  faculty,  and  staff.  Our  primary  research  questions  were: 

• What  is  your  current  level  of  use  of  e-mail  and  the  Internet? 

• Do  you  have  access  to  the  appropriate  technology? 

• What  are  your  objectives  in  using  e-mail  and  the  Internet? 

• What  obstacles  do  you  face  in  using  them? 

• What  performance  and  training  supports  do  you  feel  would  be  helpful? 

Seven  teachers  and  66  students  (73  in  all)  responded  to  the  questionnaire.  Some  of  the  information  guided  us  in  the 
next  phase  of  our  work,  i.e.:  developing  home  pages  for  the  School  of  Education,  our  doctoral  program,  and  our  division, 
and  developing  additional  job  aids  and  other  types  of  learner  support.  We  have  enumerated  our  key  Findings  below. 

We  are  online.  Fifty-nine  respondents  already  had  e-mail  accounts  on  one  of  the  university  computers.  About  half 
learned  to  use  e-mail  in  classes;  the  rest  learned  on  their  own  or  from  friends. 

We  are  mobile.  Those  who  use  e-mail  nearly  every  day  connect  from  home,  whereas  those  who  use  it  weekly  connect 
from  home  or  from  the  university  computer  labs.  60  of  the  73  respondents  have  modems  and  computers. 

We  have  good  e-mail  access.  Most  felt  their  e-mail  service  was  reliable  and  accessible,  and  that  their  system  permitted 
them  to  download  and  save  files. 

We  have  different  reasons  for  using  e-mail.  When  asked  to  rate  the  usefulness  of  a variety  of  reasons  for  using  e-mail, 
the  responses  tended  to  cluster  into  four  groups:  local  access  and  communication,  scholarly  research,  collaboration,  and 
information  dissemination. 
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• Very  useful:  use  e-mail,  locate  instructional  materials,  transfer  files  from  remote  locations,  consult  with  classmates, 

instructors,  and  advisor;  , - 

• Moderately  useful:  do  literature  searches,  access  electronic  publications,  articles,  and  scholarly  journals,  transfer 

information  between  home  computer  and  university  account,  and  organize,  store,  and  print  information  in  student 

account;  . u . 

• Somewhat  useful:  observe  other  Internet  sites,  participate  in  electronic  discussion  groups,  collaborate  with  scholars 

from  other  universities  worldwide; 

• Less  useful:  author  and  publish  electronic  documents  with  hypertext  links  to  remote  sources. 

Responses  were  fairly  evenly  divided  into  these  four  groups,  indicating  a wide  range  of  utilization  of  Internet  resources. 

We  have  a mix  of  challenged  and  experienced  users  with  different  needs.  Respondents  were  asked  to  identify  the 
challenges  they  encountered  using  e-mail.  From  those  responses,  we  separated  the  early  adopters  from  a group  of 
respondents  who  seemed  to  have  more  difficulties  with  the  technology.  The  two  groups  were  then  compared  as  they 
expressed  preferences  for  the  eight  training  and  performance  support  aids  which  we  were  considering  developing.  The 
rankings,  reported  in  Table  2 below,  were  very  different  for  each  group. 


Early  Adopters 

Reluctant  Users  of  the 
Technology 

1 . online  tutorials, 

2.  workshops, 

3 . paper  tutorials, 

4.  interactive  demonstrations, 

5.  booklets, 

6.  classes, 

7.  brochures, 

8.  help  from  graduate  assistants. 

1 . help  from  graduate  assistants, 

2.  workshops, 

3.  classes, 

4.  paper  tutorials, 

5 . interactive  demonstrations, 

6.  brochures, 

7.  booklets, 

8.  online  tutorials. 

Table  1 . Preference  rankings  for  different  kinds  of  performance  support,  across  two  user  profiles. 


It  is  evident  that  new  users  seek  face-to-face  instruction:  either  individual  attention  by  graduate  assistants  in  the 

laboratory,  or  one-  to  two-hour  workshops  and  formal  classes. 

The  early  adopters  included  experienced  and  frequent  users.  This  group  seems  comfortable  with  tutorials,  both  online 
and  paper,  whereas  one-on-one  assistance  from  graduate  assistants  was  at  the  bottom  of  the  list— possibly  because 
graduate  assistants  working  in  the  lab  are  not  formally  trained  or  certified  in  the  use  of  the  Internet! 

Interestingly,  both  groups  ranked  one-  to  two-hour  workshops  very  highly  (rank=2),  whereas  formal  classes  were  ranked 
third  from  the  top  by  the  challenged  group,  and  third  from  the  bottom  by  the  frequent  users. 

In  an  effort  to  uncover  users’  perceptions,  we  asked  survey  respondents  a number  of  Likert-style  attitude  questions. 
Responses  often  identified  "reluctant  users"  are  reported  in  Table  2 below. 
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3.5  I feel  intimidated  by  the  techno-gurus  in  the  program. 

3.3  I feel  I can  accomplish  the  same  thing  with  mail  and  phone  calls. 

3.3  I'm  the  type  that  needs  a lot  of  hand-holding. 

3. 1 I'm  concerned  that  I'd  have  to  learn  too  much  technical  jargon  and  commands  to  make  it 
worthwhile. 

3.0  I don't  have  the  time  to  learn  e-mail. 

3.0  I would  like  to  see  more  use  of  E-mail  in  my  classes. 

2.6  I think  E-mail  is  worth  the  time  and  trouble. 

3.6  Getting  access  from  home  is  too  expensive. 

2.4  Learning  to  use  E-mail  will  pay  off  for  me  professionally. 

2.3  I think  E-mail  can  help  me  stay  in  touch  with  people  better. 

4.0  I have  a hard  time  expressing  my  thoughts  in  writing. 


Table  2.  Attitude  responses  of  ten  survey  respondents  who  showed  a reluctance  to  use  e-mail  and  Internet 
resources,  with  5 converting  to  "strongly  agree",  3 to  "undecided",  and  1 to  "strongly  disagree." 

These  responses  from  reluctant  users  reflect  a number  of  underlying  factors — self-concept,  social,  and  practical — that 
contribute  to  a person's  choice  to  use  e-mail  and  the  Internet  in  their  day-to-day  work.  In  spite  of  users'  reluctance  to 
make  use  of  the  technology,  their  responses  indicate  a complex  relationship  with  that  technology.  Discussion  of  these 
issues  is  continued  in  the  section  below. 

Further  Insights  into  User  Perceptions 

Since  the  summer  of  1993,  Brent  has  been  using  e-mail  as  a mode  of  communication  within  his  classes  (Lowry, 
Koneman,  Osman-Jouchoux,  & Wilson,  1994;  Wilson,  Lowry,  Koneman,  & Osman-Jouchoux,  1994).  Students  receive 
e-mail  accounts  and  are  encouraged  to  communicate  with  each  other  and  with  the  instructor.  Reactions  to  readings, 
discussions  of  the  text,  class  announcements,  and  questions  to  the  instructors  are  all  commonly  communicated 
electronically. 

In  the  spring  of  1995,  Brent's  doctoral  seminar  went  the  next  step  and  began  using  the  Internet  seriously  as  a research 
tool  and  as  an  object  of  study.  The  thirteen  class  members  were  about  evenly  divided  between  students  with  an  interest 
in  technology  and  other  students  who  took  the  course  as  a program  requirement.  The  class  spent  nearly  half  the  classtime 
exploring  the  Internet,  developing  homepages,  conducting  searches,  downloading  files,  and  participating  (electively)  in 
listserv  discussion  groups. 

A group  interview  with  the  13  students  was  conducted  nine  weeks  into  the  semester.  This  group  interview  served  as  an 
important  source  of  information  concerning  students'  motives,  attitudes,  and  perceptions  of  the  Internet  as  a learning  tool. 
The  following  questions  were  sent  out  on  e-mail  in  anticipation  of  the  interview,  and  guided  our  discussion  (see  Table  3). 


Questions  to  New  Users  of  the  Internet 

— Did  I embrace  or  resist  the  technology?  Why? 

—How  did  my  thinking  change  as  I learned  to  use  the  Internet?  Were  there  changes  in  my  mental  models?  Changes  in 
attitudes?  Were  there  any  breakthroughs  or  "aha"  experiences? 

— What  kinds  of  supports  or  experiences  were  most  helpful  to  me  in  learning  to  take  full  advantage  of  e-mail  and  the 
Internet?  Why? 

—In  what  ways  can  the  Internet  be  used  for  learning  purposes?  What  is  the  potential  versus  the  reality?  How  can  the 
potential  be  realized? 

—Has  my  experience  with  the  Internet  influenced  the  way  I think  about  technology  and  education?  What  do  I know 
now  that  I did  not  even  think  about  three  weeks  ago  in  terms  of  how  I might  use  technology  for  education? 

— Can  tools  like  the  Internet  help  us  create  collaborative,  knowledge-  building  learning  communities?  Has  my 
experience  with  the  Internet  influenced  how  I think  about  education  and  learning  in  general? 

— Are  there  concerns  about  potential  abuses  of  technology  and  education?  How  can  we  use  the  Internet  in  ways  that 
affirm  people's  individuality  and  humanity? 

Table  3.  Questions  put  to  members  of  the  doctoral  seminar. 
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We  have  organized  student  comments  around  several  key  factors  that  affect  students'  choices  to  use  or  avoid  e-mail  and 
the  Internet.  Names  have  been  changed  to  protect  the  anonymity  of  responses.  (For  a further  analysis  of  features  of  the 
Internet  which  inhibit  of  encourage  interaction,  see  Ryder,  Wilson,  & Myers,  1996). 

1 . Clear  payoff.  There  needs  to  be  some  compelling  need  for  students  to  engage  in  the  discomfort  attending  the 
learning  of  new  technologies.  Says  one  student:  "How  is  this  gonna  help  me  down  the  road?"  Jon  used  the  example  of 
his  wife  pressuring  him  to  use  Quicken  for  their  personal  finances.  "I  have  always  known  exactly  what  I have  in  the 
bank  within  $5.  What  do  I need  Quicken  for?" 

A goal  to  use  the  technology  can  come  from  a class  assignment  or  expectation,  but  also  from  students  themselves. 

Carl  described  a personal  learning  goal  he  created:  To  create  an  electronic  portfolio  showcasing  his  work.  A portfolio  is  a 
major  requirement  of  the  doctoral  program;  Carl  decided  to  do  his  electronically.  With  this  goal  in  mind,  Carl's  heavy 
participation  on  the  Internet  had  a focus  and  purpose. 

2.  Overcoming  technophobia  and  technophatigue.  Technophobia  is  common  because  it  seems  there  is  always  some 
new  technology  demanding  to  be  learned.  Technophobia  typically  is  triggered  when  two  things  are  coupled: 

• a belief  in  the  value  or  necessity  of  learning  and  using  a new  technology; 

• feelings  of  incompetence  or  inability  to  learn  the  new  technology. 

Ironically,  the  higher  someone  values  the  technology,  the  more  extreme  the  phobia  can  become. 

Many  people  have  deeply  ingrained  feelings  of  incompetence  regarding  technology.  These  feelings  of  inadequacy,  when 
reinforced  by  past  experiences  of  failure,  can  lead  to  a condition  of  "learned  helplessness"  where  the  person  quits  trying 
and  becomes  entrenched  in  the  avoidance  of  technology.  Persisting  avoidance  behavior  only  widens  the  knowledge  gap, 
leading  to  a v cious  cycle  of  avoidance,  growing  incompetence,  and  feelings  of  inadequacy. 

Several  students  in  our  interview  reported  a need  for  unpressed  time  to  dabble,  observe  others,  and  become  comfortable 
with  the  technology.  Deb  said,  "It  takes  me  awhile  to  process,  then  when  I get  it — [snaps  finger] — I go  for  it."  Kate's 
strategy  is  similar:  "I  soak  myself  in  it  for  a long  time,  then  1 11  use  it  after  I m comfortable.-  Kate  reported  that,  because 
her  learning  style  is  conceptual  and  highly  visual,  she  is  developing  her  own  job  aids  to  support  her  performance.  Alta 
has  a friend  who  lives  close  by,  just  across  town.  But  she  enjoys  communicating  back  and  forth  with  this  friend  via  e- 
mail.  It’s  a safe  way  to  practice  her  new  skill.  "My  daughter  and  son  are  both  in  college.  I found  out  they  were 
communicating  over  e-mail  and  it  made  me  curious.  If  it  had  been  an  assignment,  though,  I would  have  thought 
differently  about  it.” 

3.  Cultural/personal  compatibility.  Technology  occasionally  conflicts  with  people's  learning  styles,  self-concepts, 
and  lifestyles.  Some  people  actively  resist  technology  on  grounds  of  principle,  believing  the  technology  influences  their 
lives  in  negative  ways.  To  address  this  concern,  students  need  skills  and  tools  to  manage  the  complexity  and  retain  a 
sense  of  control  over  the  technology  (and  not  vice  versa). 

Frank,  for  example,  commented  on  the  problem  of  information  overload  on  the  net.  He  sees  the  Internet  as  an  open- 
ended  tool,  but  "the  World-Wide  Web  is  a garage  sale — There's  so  much  junk  out  there!  I have  to  be  selective  in  how  I 
spend  my  time  and  what  resources  I use." 

Deb  described  some  of  the  interpersonal  dynamics  involved  in  e-mail  communications.  She  found  it  amazing  that  the 
co-worker  in  the  next  cubicle  would  send  e-mail  to  send  simple  messages  rather  than  just  saying  "Hey!"  across  the 
partition.  She  also  described  times  when  people  would  post  messages  laced  with  an  emotional  undercurrent,  concerning 
job  conditions  or  personal  items.  Deb  gets  a message  like  this  and  replies,  "We  need  to  talk.”  At  that  point,  the 
communication  returns  to  face-to-face. 

4.  Proper  scaffolding.  People  need  a "scaffold"  or  support  structure  in  place  as  they  engage  in  complex  performances 
outside  their  normal  repertoire  of  skills.  In  the  case  of  the  Internet,  a complete  scaffolding  system  seems  to  include: 

• Supportive,  non-judgmental,  social  support  system.  Scott  noted  a problem  with  adult  learners.  We  tend  to  assume 
that  everyone  knows  how  to  do  basic  things.  "I  shouldn't  have  to  ask  for  help — I’m  a graduate  student!"  Then 
we're  embarrassed  when  don't  know  what  we’re  doing,  and  we're  afraid  to  ask.  "You  don’t  wanna  look  stupid,  so 
you're  not  going  to  venture  out...” 

• Lots  of  hand-holding.  New  users  of  technology  can  easily  get  overwhelmed  by  the  complexity  and  newness  of  it 
all.  Greg  complained,  "We  don't  know  what  we  don't  know."  Carl  suggested  that  technology  be  mastered  in  small 
groups  of  2-3  learners,  maximum.  People  need  personal  attention  and  hand-holding  that  is  not  possible  in  a full 
class.  Five  minutes  into  a full-class  demo,  some  students  get  lost  but  won't  say  anything,  because  they  feel  there 
is  nothing  that  can  be  done  in  such  a large  group. 

• Freedom  from  technical  hurdles.  Richard  described  his  attitude  towards  obstacles  in  using  technology.  "You  hit  a 
barrier  once — jump  over  it.  You  hit  it  a second  time,  remove  the  barrier."  Richard  described  how  having  a clear 


application — a decision  to  begin  implementing  e-mail  in  his  own  classes — prompted  him  to  make  steady  progress 
toward  using  the  technology.  "It’s  like  the  movie,  ’What  About  Bob' — I’m  taking  baby  steps  but  I'm  getting 
there.0 

• Access.  One  student  had  serious  problems  initially  because  he  did  not  own  a computer  and  modem,  and  had 
difficulty  getting  on  campus.  This  lack  of  access  initially  affected  his  attitudes  and  learning  of  Internet  skills. 

Finding  a good  time  to  get  online  is  a problem  for  Alta.  "If  you  look  at  my  times,  you'll  see  that  I'm  on  at  5:00  am 
I can't  get  on  during  the  day — the  modem  pool  is  overloaded.  Sunday  night  is  impossible."  So  she  logs  on  before  work, 
which  leaves  her  little  time  to  reflect  and  respond  to  mail,  even  when  she  is  inclined  to  do  so. 

Jack  had  similar  access  problems.  He  lives  outside  the  local  dialing  area,  but  eventually  learned  how  to  telnet  from  one 
computer  to  another.  This  made  communication  from  home  to  the  university  economically  feasible. 

5.  Finding  a voice  and  having  something  to  say.  Lynn  described  some  of  the  concerns  that  are  keeping  her  from 
finishing  her  homepage.  "The  issue  of  progress  isn't  with  the  technology.  There's  lots  of  supports — We  know  how  to 
get  help.  My  problem  is  the  CONTENT,  not  the  technology.  I still  don't  know  what  I want  to  put  into  my  homepage, 
and  how  I want  it  to  look." 

Frank  agreed  with  Lynn.  "I  used  the  'private'  command  to  lock  up  my  homepage  until  I'm  ready  to  show  it  off.  I'm 
spending  time  developing  the  text  document.  I want  to  make  sure  my  stuff  is  substantive  and  cogent.  I don't  want  the 
whole  world  thinking  I'm  a fool!" 

Followup  with  Students  and  Faculty 

To  examine  possible  shifts  in  students'  perceptions  and  Internet  use  since  the  Spring  term,  a follow-up  interview  via  e- 
mail  was  conducted  during  the  fall  of  1995.  A short  questionnaire  was  sent  to  the  thirteen  students  who  had  been  enrolled 
in  the  spring  doctoral  seminar. 

The  students  were  asked  to  re-assess  their  attitudes,  perceptions,  and  use  of  e-mail  and  the  Internet  in  relation  to  their 
academic  work.  Seven  of  the  thirteen  students  replied.  Their  comments  reveal  two  clusters  of  students:  Some  students 
have  remained  static  or  resistant  toward  technology  use,  while  others  dynamically  appropriate  the  technology  into  their 
research  and  communications.  Students  in  the  second  group  have  entered  into  a dynamic  relationship  with  the 
technology,  with  their  personal  and  professional  growth  stimulated  by  new  resources  and  tools  available  on  the  Internet. 

This  dynamic  growth  is  reflected  by  an  extended  use  of  the  Internet  as  a research  tool.  Kate  experienced  a major  shift  in 
how  she  thought  about  and  used  the  Internet:  "What  I wasn’t  tuned  into  at  first  was  my  potential  role  on  the  Net.. .that  I 
could  be  a resource  for  others.. .that  I could  have  a site  and  offer  my  wares  to  the  world."  Kate  uses  the  Internet  almost 
daily  to  communicate  within  the  virtual  community.  She  gives  credit  to  a professor  who  provided  support  as  she . 
underwent  this  change  process. 

Another  equally  enthusiastic  adopter  is  Scott.  His  use  of  the  Internet  has  been  stimulated  by  his  experiences  outside 
the  department.  He  commented,  "...I  conferred  with  an  acquaintance  in  Britain  more  frequently  on  e-mail  this  semester 
than  I did  with  my  classmates  at  UCD."  Scott  questions  the  department’s  support  of  the  Internet  as  a resource  for 
students.  He  is  concerned  that  some  students  perceive  the  Internet  as  not  a valued  resource  or  a place  to  participate  in  the 
larger  academic  community.  For  technical  support,  another  student,  Oprah  has  also  reached  out  to  the  broader  Internet 
community,  seeking  technical  help  from  her  daughter,  a computer  engineer  at  a university  in  another  state. 

In  contrast,  we  noted  the  work  done  by  the  Task  Force  extends  beyond  our  university  walls.  One  student  who  sees 
values  in  the  Internet  but  uses  these  resources  irregularly,  reported  that  he  would  use  locally  developed  resources  to 
develop  a homepage  at  a neighboring  university. 

Janell,  an  active  e-mail  user,  explains  why  she  doesn’t  take  advantage  of  Internet  resources: 

My  thinking  has  not  changed  re:  the  Internet  since  May.  I still  think  that  it  is  a very  time  consuming  effort  to  ' 
search  and/or  monitor  the  Internet  for  my  learning  needs.  I realize  that  the  vast  amount  of  information  available 
could  be  invaluable,  but  I do  not  currently  have  the  time  or  the  inclination  to  'fight'  the  technology  as  well  as 
the  effort  needed  to  'find'  what  I need. 

We  can  only  conclude  that  the  six  students  who  chose  not  to  respond  to  the  followup  questions  may  hold  similar  views. 
Person-to-person  followup  may  shed  further  light  on  these  students'  use  of  e-mail  and  the  priority  of  the  Internet  in  their 
graduate  studies. 

Recognizing  the  central  role  played  by  faculty  members  in  setting  a tone  for  a department’ s local  culture,  we  decided  to 
probe  into  faculty  attitudes.  In  spite  of  their  affiliation  with  a technology  program — or  perhaps  because  of  it! — faculty 
members  can  be  expected  to  show  varying  attitudes  toward  the  adoption  of  new  technologies.  Four  faculty  members  in 
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the  ILT  department  were  contacted  to  participate  in  a telephone  interview.  Three  of  the  four  responded.  Two  of  three 
interviews  were  conducted  via  phone  and  one  via  an  e-mail  interview. 

The  interviews  revealed  different  levels  of  use  and  interest  toward  the  Internet.  One  faculty  member  has  been  using  the 
electronic  medium  to  communicate  globally  with  an  entire  class,  establishing  online  discussion  groups  and  encouraging 
individual  communication  among  class  members  and  their  professor.  Students  are  encouraged  but  not  required  to  use 
online  resources  for  class  projects.  Another  faculty  member  also  encourages  students  to  communicate  via  e-mail, 
requiring  it  for  some  classes.  But  non-adopters  can  be  found  among  faculty  members,  too.  The  third  faculty  member  felt 
the  Web  is  not  a credible  place  to  either  place  a paper  or  to  find  published  research  of  fellow  academics.  Using  the 
Internet  is  perceived  to  be  a cumbersome,  time  consuming  activity. 

Overall,  student  and  faculty  comments  were  consistent  with  survey  responses  in  showing  the  diverse  responses  the 
Internet  can  provoke.  Some  members  of  the  local  culture  make  a conscious,  reflected  choice  to  avoid  these  technologies, 
while  perhaps  most  make  cautious,  calculated  forays  as  time  and  expected  payoff  allow.  For  the  Task  Force,  dedicated  to 
continuing  the  assimilation  of  the  Internet  culture  into  the  local  culture,  these  findings  provide  a clear  mandate  for 
respecting  diverse  points  of  view  and  for  providing  comprehensive  support.  We  are  at  beginning  stages  in  providing  that 
support,  as  sported  in  the  section  below. 

Where  We  Are  Now:  A Progress  Report 

The  response  of  the  Task  Force  to  surveys  and  interviews  have  taken  a number  of  forms,  including: 

— developing  information  brochures  describing  local  resources; 

— creating  a student  distribution  list  for  job  announcements,  etc.; 

— maintaining  and  improving  departmental  homepages  and  resources; 

— a continuing  research  agenda  related  to  adoption  and  change. 

ILT  faculty  members,  in  spite  of  their  diverse  views  toward  the  Internet,  have  responded  in  turn  with  a number  of 
supportive  gestures,  including: 

— in  many  cases,  encouraging  e-mail  use  within  classes  and  between  advisors  and  students; 

— establishing  greater  presence  of  telecommunications,  distance  learning,  and  Web  authoring  in  curriculum  offerings; 
—inserting  an  e-mail  policy  statement  into  the  ILT  student  handbook,  mandating  regular  access. 

Overall,  the  local  culture  is  moving  toward  the  Internet,  but  we  are  far  from  assimilating  the  two  cultures.  E-mail  us 
is  becoming  routinized,  but  using  the  Internet  varies  dramatically,  with  faculty  differences  continuing  to  play  a key  role. 
Internet  use  has  not  yet  transcended  individual  courses  to  become  a habit  among  community  members  in  general.  A 
number  of  concerns  remain,  including: 

— concerns  with  equity  and  access  from  labs,  work,  and  home; 

— varying  expectations  among  faculty  and  courses; 

— lack  of  a graphic  interface  (SLIP/PPP)  from  home,  resulting  in  different  standards  for  software  and  interface; 

— continued  difficulty  getting  connected  from  home; 

Increasing  forays  are  being  made  toward  integrating  Internet  resources  into  coursework.  Most  recently,  a course  is  being 
planned  for  the  spring  that  will  be  taught  primarily  via  e-mail  and  the  Web. 

A significant  factor  affecting  the  adoption  of  Internet  resources  within  the  School  of  Education  is  the  competition  from 
a School-sponsored  e-mail  system  called  Colorado  Educators  Online  or  CEO.  CEO  is  based  on  the  First-Class  BBS 
system  and  allows  access  to  listservs  but  not  the  Web.  CEO  is  used  as  the  primary  e-mail  system  by  Education  faculty 
members  and  is  promoted  heavily  to  school  districts  within  the  Denver  metro  area.  A significant  advance  has  recently 
been  made  which  allows  access  to  CEO  via  Netscape  on  the  Web.  We  are  hopeful  that  once  students  and  faculty  invest 
in  SLIP  access  from  home,  the  Web  will  be  more  easily  integrated  into  their  regular  e-mail  activities. 

The  Internet  Task  Force  continues  in  its  goals  to  support  the  integration  of  Internet  resources  in  the  School.  We 
struggle  to  maintain  a dual  focus  of  maintaining  Web  pages  while  at  the  same  time  promoting  and  seeking  to  understand 
the  adoption  process.  This  semester,  we  have  begun  splitting  these  two  responsibilities  among  two  iub-groups,  one 
charged  with  Web  maintenance  and  the  other  charged  with  engaging  in  change  research. 

Conclusions  and  Recommendations 

Theory  and  practical  knowledge  about  how  to  use  the  Internet  lags  behind  the  technology  itself.  This  is  to  be  expected. 
Cuban  (1986)  surveyed  the  history  of  technologies  used  in  education  and  found  a pattern  that  included  initial  stages  of 
excitement  and  hype,  followed  by  predictable  backlashes  and  retrenchment.  Initial  uses  of  a new  technology  tend  to 
mirror  existing  educational  forms  and  practices — e.g.,  traditional  classrooms,  lectures,  and  control  structures.  Eventually 
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we  expect  to  see  greater  departure  from  traditional  educational  forms,  with  the  Internet  enabling  new  paradigms  and 
approaches  to  learning  (Blurton,  1994;  Lemke,  1993). 

To  understand  people’s  use  and  resistance  of  the  Internet,  we  have  felt  a need  to  go  beyond  traditional  adoption  models 
(e.g.,  Rogers,  1995).  The  coming  role  of  e-mail  and  the  World-Wide  Web  in  people’s  lives  may  better  be  understood 
when  seen  in  systemic,  organismic,  and  chaotic  terms  (Bateson,  1972;  Kelly,  1994).  We  are  presently  drawing  on 
theoretical  frameworks  of  biologists,  cognitive  psychologists,  and  anthropologists  as  we  seek  to  understand  the  learning 
potential  of  the  Internet. 

Our  observations  have  also  heightened  our  awareness  of  the  difficulty  in  bringing  two  cultures  together.  Users  must 
surpass  a number  of  barriers  as  they  become  initiates  into  the  Internet  culture.  We  found  the  technical  barriers  to  be  the 
easiest  ones  to  solve.  But  there  are  many  more  that  are  rooted  in  culture,  lifestyles,  learning  styles,  paradigms,  and 
comfort  zones.  These  cultural  aspects  of  Internet  use  provide  rich  opportunities  for  further  research,  theory  development, 
and  guidance  for  practice. 

David  Perkins  (1996)  notes  an  important  kind  of  learning  that  is  not  often  acknowledged  in  formal  school  settings: 
Knowing  your  way  around.  Just  as  we  knew  our  way  around  our  neighborhood  as  children,  Perkins  suggests  that 
knowing  our  way  combines  a number  of  kinds  of  knowing,  including: 

...having  a sense  of  orientation,  recognizing  problems  and  opportunities,  perceiving  now  things  work  together, 
possessing  a feel  for  the  texture  and  structure  of  the  domain.  It  encompasses  not  just  explicit  but  tacit 
knowledge,  not  just  focal  awareness  but  peripheral  awareness,  not  just  a sense  of  what’s  there  but  whafs 
interesting  and  valuable....  [KJnowing  your  way  around  resonates  with  the  notion  of  a learning  environment,  (p. 
vi) 

Knowing  our  way  around  can  be  critical  in  a variety  of  domains: 

We  can  speak  sensibly  of  knowing  your  way  around  the  stock  market,  playing  baseball,  and  any  discipline,  for 
instance  physics  or  English  literature.  To  really  know  any  of  these  domains  requires  a kind  of  flexible 
orientation  to  what  things  and  places  they  contain,  what  resources  they  afford,  and  how  to  get  jobs  done.... 
(Perkins,  1996,  p.  vi) 

We  agree  with  Perkins.  People  who  learn  their  way  around  the  Internet  are  learning  more  than  facts,  rules,  and  * 
procedures.  As  Perkins  notes,  being  able  to  "get  around”  an  information-rich  environment  is  closely  related  to  mastering 
the  complexity  of  modem  disciplines  (c/.  Kelly,  1994).  Through  the  Internet  and  similar  collaborative  learning 
environments,  people  can  become  enculterated  into  new  disciplines  and  knowledge-building  communities  (Scardamalia  & 
Bereiter,  1994).  Students  facing  limited  resources  in  their  local  environments  (e.g.,  an  under-funded,  inner-city  school) 
potentially  can  reach  out  to  unlimited  resources  on  the  Internet.  The  possibilities  are  staggering:  The  Internet  has  the 
potential  to  be  an  "equalizer"  that  introduces  novice  outsiders  into  a rich  community  full  of  resources  and  expertise. 

Of  course,  something  is  missing  from  the  picture.  Impoverished  or  dysfunctional  local  communities  will  not  prepare 
students  with  the  metacognitive  and  dispositional  qualities  needed  to  take  advantage  to  a rich  virtual  environment. 
Students  who  have  never  "seen  the  light"  of  day  cannot  be  expected  to  rush  outside  and  begin  playing  football  on  the 
lawn.  Even  so,  the  Internet  may  come  to  serve  as  a "Trojan  horse"  in  such  limited  local  environments,  subverting  the 
established  order,  providing  a second  voice  or  second  perspective  that  will  stimulate  change  at  the  local  level. 

Even  within  healthy  local  environments,  providing  clear  guidance  for  practitioners  is  a challenge  as  they  try  to 
incorporate  the  Internet.  Understanding  the  factors  affecting  use,  however,  should  help  educators  get  a better  grasp  of 
methods  to  bridge  the  gap  between  the  two  cultures.  In  this  light,  we  offer  the  following  recommendations  to  educators 
using  the  Internet: 

1 . Provide  experiences  which  guarantee  first-time  success  (e.g.,  don't  require  users  to  install  and  configure  their  own 
software). 

2.  Provide  ample  scaffolding  from  initial  diiect  hand-holding  to  on-line  job  aids  and  help  facilities,  to  on-going  human 
(peer)  support.  Everyone  in  our  study  benefited  most  significantly  from  direct,  live,  person-to-person  support. 

3.  Integrate  Internet  resources  by  providing  authentic  tasks  that  provide  students  legitimate  reasons  to  use  the 
technology. 

4.  Integrate  Internet  resources  into  traditional  classrooms,  but  also  cultivate  informal,  student-directed  uses.  Internet 
adoption  will  be  most  successful  in  organizations  that  encourage  independent  and  collaborative  inquiry,  student- 
directed  learning,  and  professional  responsibility. 
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5.  Encourage  adoption  of  Internet  cultural  practices  through  a variety  of  incentives,  policies,  and  practices,  but  keep  to 
a minimum  explicit  mandates  and  requirements.  Seek  to  create  an  atmosphere  of  expected  and  natural  Internet 
participation  without  the  feeling  of  coercion. 

6.  Encourage  students  to  represent  themselves  to  a world-wide  community  of  learners  by  offering  unique  contributions 
to  society's  knowledge  base.  This  single  strategy  affords  ownership  and  empowerment— the  learner  has  made  her 
presence  known  in  cyberspace.  She  is  attached  to  the  World-Wide  Web! 
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INTRODUCTION 

Technology  tools  have  been  used  in  schools,  corporations,  and  government  agencies  to  extend  or  enhance  both 
physical  and/or  cognitive  human  capabilities  in  work  and  study.  In  education,  they  have  been  used  to  facilitate  and 
revolutionize  teaching  and  learning.  Distance  learning  is  one  of  many  educational  initiatives  that  has  greatly  benefited 
from  the  advent  of  recent  technology  tools,  such  as  electronic  conferencing  systems,  resource  sharing  systems,  and  group 
access  support  systems.  The  relatively  low-cost  electronic  conferencing  system  is  proving  to  be  a powerful  and 
increasingly  popular  medium-either  for  totally  ’online1  courses  or  for  more  conventional  in-classroom  and  distance 
learning  courses  (Verdejo  & Cerri,  1994).  Electronic  conferencing  promotes  student-centered  learning.  That  is,  it  no 
longer  emphasizes  the  teacher  as  gospel  or  the  primary  source  of  knowledge.  Different  kinds  of  electronic  conference 
software  packages  provide  technical  and  pedagogical  support  for  organizing  and  structuring  conference  topics  and  message 
retrieval. 

The  present  study  describes  how  VaxNotes,  an  electronic  conference  software,  was  used  at  a large  midwest 
University  in  a graduate  distance  learning  course  ’’Interactive  Technologies  for  Learning”.  In  that  course,  students’ 
electronic  discussions  and  knowledge  construction  as  well  as  instructors'  methods  for  organizing  the  electronic  conference 
in  distance  learning  course  were  saved  and  analyzed.  The  study  revealed  and  discussed  patterns  of  students'  electronic 
discussion  and  knowledge  construction  practices.  Additionally,  this  study  reviewed  and  analyzed  various  ways  of 
organizing  the  electronic  conference  and  mentoring  in  this  distance  learning  course.  In  effect,  this  study  provided  insights 
into  how  to  employ  electronic  conferencing  technology  in  facilitating  teaching  and  learning  by  informing  teachers  of 
what  technology  was  used  in  the  classroom  and  how  it  was  used  as  well. 

It  is  hoped  that  the  study  will  be  informative  and  will  perhaps  inspire  other  instructors  to  employ  social  and 
cognitive  learning  theories  in  an  electronic  conference.  The  work  presented  here  can  be  a springboard  for  other  researchers 
to  use  social  and  cognitive  learning  theories  as  the  theoretical  framework  for  continued  research  in  computer  conferencing. 

THEORETICAL  FRAMEWORK 

Vygotsky's  learning  theory  and  theories  of  cognitive  and  constructive  learning  provide  a helpful  theoretical 
framework  for  the  present  study. 

The  conceptual  framework  of  Vygotsky’s  theory  provides  a good  basis  for  understanding  learning  as  a process  of 
social  negotiation  and  collaborative  sense  making,  and  mentoring  as  an  effective  technique  to  assist  students  in 
collaborative  activities  and  knowledge  construction.  One  of  the  key  concepts  in  Vygotsky’s  theory  is  the  zone  of 
proximal  development.  Vygotsky  defined  this  as  "the  distance  between  the  actual  developmental  level  as  determined  by 
independent  problem  solving  and  the  level  of  potential  development  as  determined  through  problem  solving  under  adult 
guidance  or  in  collaboration  with  more  capable  peers"(1978,  p 86).  This  definition  expresses  Vygotsky's  notion  that 
what  children  can  do  with  the  assistance  of  others  might  be,  in  some  sense,  even  more  indicative  of  their  mental 
development  than  what  they  can  do  alone  (Meichenbaum,  et  al.  1985).  Their  achievements  in  a supportive  social 
environment  make  the  largest  contribution  to  their  cognitive  development  ^Fowler  & Wheeler,  1995). 

A related  important  concept  in  Vygotsky’s  theory  is  his  idea  that  intellectual  development  takes  place  between 
people  before  internalization  takes  place.  Vygotsky  contended:  "Any  function  in  the  child’s  cultural  development  appears 
twice,  or  on  two  planes.  First  it  appears  on  the  social  plane  and  then  it  appears  on  the  psychological  plane.  First  it 
appears  between  people  as  an  interpsychological  category,  and  then  within  the  child  as  an  intrapsychological 
category "(1981).  That  is,  our  social  and  cognitive  development  occurs  when  we  are  interacting  with  peers  and  experts. 
The  interactions  with  peers  and  experts  in  the  social  content  and  context  transform  the  interpersonal  process  into  the 
intrapersonal  process.  Socially  displayed  values,  theories,  and  ideas  are  thus  internalized  in  individuals  through  this 
transformation. 

From  this  point  of  view,  instruction  is  most  effective  when  it  is  in  a form  of  discussions  or  dialogues  wherein 
learners  can  interact  with  peers  and  adults  or  mentors  who  challenge,  support,  and  scaffold  their  learning.  As  sociological 
researchers  point  out,  instruction  should  take  place  in  a social  environment,  in  which  learners  use  socially  mediated  and 
intellectual  tools  to  achieve  cognitive  development  (Rogoff,  1990).  Here,  the  instructor  designs  learning  activities  in  the 
zone  of  proximal  development.  This  zone  is  neither  a property  of  the  child  nor  of  the  interpsychological  functioning 
alone,  but  is  jointly  determined  by  the  child's  level  of  development  and  the  form  of  instruction  involved.  The  challenge 
to  educators  is  to  develop  forms  of  instruction  so  that  interpsychological  functioning  can  be  structured  to  maximize  the 
growth  of  intrapyschological  functioning  (Wcrtsch,  1990). 

An  electronic  conference  in  distance  learning  courses  or  regular  courses  can  be  viewed  as  a device  through  which 
interactions  between  learners  and  instructors  can  take  place.  By  using  the  electronic  conference,  they  both  can  voice  their 


opinions  and  reflect  on  their  learning.  As  indicated,  these  interactions  among  learners  and  instructors  increase 
interpsychological  and  intrapsychological  activities  and  maximize  individual's  cognitive  growth  and  development. 

The  social  aspect  of  learning  and  intellectual  development  is  one  part  of  the  theories  guiding  the  present  study. 
The  other  part  is  the  view  of  learning  as  a constructive  process  of  engaging  in  self-regulated,  constructive,  and  reflective 
activities.  Quite  a few  theorists  challenged  the  traditional  role  of  the  teacher  and  the  student.  They  argued  that  the  role  of 
teaching  and  other  instructional  media  is  shifting  from  one  that  seeks  to  maximize  the  communication  of  fixed  content 
and  skills  to  one  in  which  students  are  led  to  experience  the  knowledge  construction  process  (Knuth  and  Cunningham, 
1993).  From  this  perspective,  students  construct  interpretations,  appreciate  multiple  perspectives,  develop  and  defend 
their  own  positions  while  recognizing  others,  and  become  aware  of  and  able  to  manipulate  the  knowledge  construction 
process  itself.  That  is,  students  are  no  longer  passive  recipients  of  bodies  of  knowledge,  but  are  actively  involved  in  the 
knowledge-building  process,  in  which  they  discuss,  elaborate  upon  concepts,  and  devise  relationships  among  them,  and  in 
which  they  view  multiple  perspectives  on  concepts  or  issues,  and  generate  their  understanding  based  on  prior  knowledge 
and  current  understanding. 

"While  the  individual  learner  is  the  only  one  who  can  construct  his  or  her  unique  understanding  of  the  world, 
this  understanding  emerges  in  a social  context"  (Cunningham,  Duffy  & Knuth,  p.  25.  1993).  We  view  the  electronic 
conference  in  the  distance  learning  course  as  the  social  context  from  which  individual  understanding  is  emerged.  The 
electronic  conference  provides  a space  for  students  to  exchange  ideas,  discuss  issues,  and  join  efforts  in  searching  for 
solutions  to  problems.  It  enables  asynchronous  collaboration  in  the  distance  learning  course. 

Learning  is  not  only  active,  constructive  and  cumulative  but  also  a goal  directed  process.  It  is  goal  oriented  in 
that  learning  is  most  likely  to  be  successful  if  the  learner  is  aware  of  the  goal  towards  which  he  or  she  is  working  and  if 
the  learner  holds  expectations  that  are  appropriate  for  attaining  the  desired  outcome  (Schuell,  1988).  Learning  requires 
learners  to  engage  in  collaborative  activities  and  in  self-regulated  and  reflective  activities  as  well.  The  distance  learning 
course  in  this  study  provided  such  opportunities  and  experiences  for  learners.  While  students  were  participating  in  the 
electronic  discussion,  they  engaged  not  only  in  interaction  and  collaboration  with  peers,  but  in  such  activities  as  goal- 
setting, planning,  and  monitoring  as  well. 

John  Dewey  once  argued  that  human  intelligence  and  all  learning  are  cultivated  through  reflective  thinking 
(1933).  When  individuals  examine  and  test  their  ideas  with  a purpose,  they  are  better  able  to  use  their  knowledge  in 
informed  and  self-directed  ways.  The  electronic  discussion  net  only  enables  interaction  and  collaboration  but  promotes 
self-regulated  learning  activities  and  reflective  activities  as  well.  Reading  and  responding  to  peers’  and  instructors'  notes 
forces  students  to  think,  form  ideas,  and  articulate  them  in  a meaningful  and  sensible  way.  Reading  the  peer’s  and  the 
instructor’s  thoughts  urges  students  to  compare  them  with  their  own  thoughts  and  ideas,  and  to  examine  their  own 
understanding  and  interpretations. 

The  reflective  activity  in  the  distance  learning  course  thus  involves  students’  self-appraisals,  reflection  on  the 
class  and  on  past  experience,  and  re-construction  of  concepts  and  ideas.  In  the  reflection  process,  existing  knowledge  is 
re-constructed  to  yield  new  perspectives.  The  essential  components  of  any  effective  learning  are  social  interactions, 
reflective  activities,  constructive  activities,  and  self-regulating  learning  activities.  It  is  under  these  theoretical  and 
epistemological  beliefs  that  the  current  study  was  conducted.  Furthermore,  these  beliefs  are  consistent  with  those 
reflected  in  the  readings  and  articles  chosen  for  this  graduate  distance  course. 


METHODOLOGY 


Setting  and  subjects 

The  study  was  conducted  in  a 16-week  graduate  seminar-"Interactive  Technologies  for  Learn ing”-~co-taught  by 
two  instructors  at  a large  midwestern  University  Spring,  1995.  The  seminar  focused  on  the  discussion  of  "the  notion  of 
the  computer  as  an  educational  tool."  It  incorporated  an  assortment  of  lectures,  demonstrations,  videos,  and  small  and 
large  group  discussion  activities  as  well  as  few  live  class  discussions  with  authors  of  selected  readings  through  CU- 
SeeMe. 

The  students  in  the  course  were  mostly  in  Masters  and  Ph.  D.  programs  such  as  Instructional  Systems 
Technology,  Educational  Psychology,  and  Telecommunications.  They  met  for  three  hours  every  Monday  during  the 
spring  semester.  Students  at  two  campus  were  connected  via  video  and  audio  distance  learning  technologies,  so  that  they 
could  see  each  other  and  their  instructors  at  both  ends.  Each  class  period  was  usually  devoted  to  a variety  of  activities 
such  as  the  instructors'  summarizing  weekly  readings,  class  discussions  of  readings,  and  students  or  instructors 
demonstrating  computer-mediated  tools  for  learning.  An  average  of  80  pages  was  assigned  for  each  week's  reading. 
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Consequently,  though  each  student  could  voice  his/her  opinions,  understandings  and  questions  about  weekly  readings 
during  a session  of  three  hours,  a thorough  discussion  of  readings  and  multiple  perspectives  of  readings  could  not  be 
achieved  during  the  allocated  class  time.  Thus,  most  of  the  discussion  about  weekly  readings  and  sharing  of  individual 
understanding  took  place  in  the  form  of  computer-mediated  communication(CMC).  In  this  case,  it  was  E-mail  and 
VaxNotes.  E-mail  was  the  primary  communication  channel  for  the  first  two  and  half  week’s  discussions.  Later  class 
discussions  gradually  migrated  into  VaxNotes. 

The  student  electronic  discussion  consisted  of  four  major  components:  instructor's  introductory  questions  and 
reading  advice,  weekly  starter  comments,  weekly  wrapper  comments,  and  participant  comments.  The  instructors  designed 
the  starter,  the  wrapper,  and  weekly  participant  roles  for  the  students.  Their  participation  in  the  electronic  discussion  was 
worth  25  percent  of  the  final  grade.  Students  assumed  a starter  or  wrapper  role  just  once  during  the  semester,  but  were 
asked  to  participate  in  some  manner  each  week. 

As  a starter , a student  might:  (1)  state  reactions  and  questions  and  make  suggestions  for  the  upcoming  readings; 
(2)  point  out  the  relationship  of  an  upcoming  topic  or  articles  to  past  lectures  or  readings;  (3)  discuss  the  position  of  a 
pioneer  in  the  field;  (4)  discuss  a recent  visit  to  a technology  center,  exhibit,  or  conference  demonstrating  computer  tools; 
(5)  attempt  to  relate  student  prior  learning  and  discussion  to  the  current  week’s  readings.  In  effect,  the  starter  role  was 
intended  to  provide  the  class  with  key  themes,  issues,  or  questions  leading  to  the  upcoming  week's  readings. 

The  wrapper  role,  on  the  other  hand,  was  meant  to  bring  some  of  the  discussed  issues  and  questions  to  some 
sense  of  closure.  Summarizing  and  synthesizing  key  points  and  themes  was  within  the  wrapper's  purview.  Therefore,  as 
a wrapper,  a student  might:  (1)  react  and  reflect  on  any  lecture,  discussion,  or  demonstration;  (2)  restate  and  reflect  on  the 
starter's  initial  points  for  that  week;  (3)  point  out  questions  and  concerns  that  have  yet  to  be  answered;  (4)  note  any 
additional  related  readings;  or  (5)  react  to  a guest  speaker's  ideas  or  a unique  class  activity. 

As  a weekly  participant  one  might:  (1)  participate  in  the  discussion;  (2)  answer  questions  and  concerns  of  other 
participants;  (3)  question  or  respond  to  a peer  at  another  site;  or  (4)  bring  to  everyone's  attention  a related  conference, 
issue,  newspaper  article,  or  grant  proposal. 


Data 

The  data  was  collected  from  both  E-mail  and  VaxNotes  discussions.  A total  number  of  408  notes  were  collected 
from  E-mail  and  VaxNotes.  Of  these,  84  notes  were  contributed  by  the  instructors.  Quantitative  analysis  was  performed 
on  all  the  data  collected;  however,  due  to  the  vast  amount  of  data,  a qualitative,  descriptive,  and  detailed  analysis  was 
conducted  on  the  discussion  of  only  two  randomly  selected  weeks  (Week  9 and  Week  12). 

The  collected  E-mails  and  VaxNotes  messages  were  coded  into  participation  categories  such  as  question, 
reflection,  discussion,  comment,  and  answer.  Additionally,  each  student  and  instructor  during  the  discussion  was  studied 
in  roles  such  as  contributor,  wanderer,  seeker,  and  mentor.  Patterns  of  the  student  electronic  discussion,  interaction,  and 
knowledge  construction  were  discussed  based  on  the  analysis  of  participation  and  role  categories.  To  further  complete 
this  teaching  and  learning  picture,  the  instructors'  contributions  were  analyzed  in  the  way  of  planning,  introductory 
questions,  contributing  toward  discussions,  commenting  on  students'  responses,  and  guiding  students'  learning. 


Framework  for  analyzing  data 

The  data  analyses  disclosed  the  relationship  and  the  nature  of  students'  and  instructors'  notes  in  the  electronic 
conference,  thereby  providing  a better  understanding  of  how  students  construct  new  knowledge  and  understanding,  and 
how  electronic  conferences  can  be  efficiently  organized  to  facilitate  learning.  The  analyses  required  not  only  an 
understanding  of  how  students  contributed  to  the  electronic  conference  but  also  a familiarity  with  the  content  area  to  help 
determine  the  role  of  each  student,  and  the  evolution  and  development  of  an  idea  or  a topic.  No  previous  analytical 
framework  or  coding  schemes,  such  as  conversation  analysis  (Schegloff,  1981)  or  protocol  analysis  (Ericsson  & Simon, 
1984),  could  be  readily  applied  to  suit  such  unique  analyses  of  data. 

For  the  purpose  of  this  study,  a specific  coding  scheme  was  constructed.  The  scheme  incorporated  both  Hatano 
and  Inagaki’s  (1991)  theory  of  group  interaction  and  Graesser  and  Person's  (1994)  theory  of  question  analysis. 

Construction  of  knowledge  through  social  interaction  can  be  observed  in  two  types  of  interaction,  horizontal  and  vertical 
(Hatano  & Inagaki,  1991).  According  to  Hatano  and  Inagaki,  both  vertical  and  horizontal  interactions  could  refer  to  the 
interactions  among  peers.  In  the  vertical  interaction , some  group  members  will  concentrate  on  looking  for  the  more 
capable  member’s  desired  answers  rather  than  contribute  to  and  construct  knowledge.  By  contrast,  in  the  horizontal 
interaction , members'  desires  to  express  their  ideas  tend  to  be  strong,  because  no  authoritative  correct  answers  are  expected 
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to  come  immediately.  Therefore,  the  members  often  express  a variety  of  ideas  and  participate  in  the  exchange  of  ideas 
which  are  likely  to  be  examined  and  elaborated  in  such  interaction  (Hatano  & Inagaki,  1991). 

The  coding  scheme  for  data  analyses  consists  of  participant  categories,  types  of  interaction,  and  note  categories 
(See  Table  1 & Table  2).  In  the  participant  category,  four  kinds  of  student  roles  were  identified,  such  as  contributor, 
wanderer,  seeker,  and  mentor.  Two  types  of  interactions  were  distinguished.  Notes  were  classified  according  to  meanings 
as  reflective  notes,  comments,  discussions,  answers,  information  sharing  notes,  scaffolding  notes,  and  questions. 

Two  main  types  of  questions  are  detected  in  the  electronic  conference.  The  first,  Type  I,  is  information-seeking 
questions  (Graesser  & Person,  1994).  When  a genuine  information-seeking  question  is  asked,  the  questioner  is  missing 
information  and  believes  the  answerer  can  supply  it  (Graesser  & Person,  p.  108).  Van  der  Meiji  (1987)  identified  several 
assumptions  underlying  a genuine  information-seeking  question:  a)  the  questioner  does  not  know  the  information  asked 
for  in  the  question;  b)  the  questioner  believes  that  an  answer  exists;  c)  the  questioner  wants  to  know  the  answer;  and  d) 
the  questioner  believes  that  the  answer  will  not  be  given  in  absence  of  the  question  (Graesser  & Person,  1994). 

The  other,  Type  II,  is  discussing  questions.  Assumptions  underlying  discussing  questions  are:  a)  the  questioner 
can  provide  some  kind  of  explanation  to  the  question,  but  he  believes  that  it  may  not  be  complete  nor  most  appropriate; 
b)  the  questioner  understands  that  there  are  no  existing  and  ready  answers  to  the  question;  c)  the  questioner  would  like  to 
seek  opinions  from  peers  or  experts;  and  d)  the  questioner  intends  to  start  a dialogue  among  peers  rather  than  ask  for 
answers.  Questions  here  are  usually  deep-level  ones  regarding  particular  research  issues  but  normally  without  definite 
answers. 

Reflective  notes  are  defined  as  any  reflective  thoughts  running  through  participant  notes.  These  can  be:  a) 
evaluation  of  the  class  and  learning;  b)  self-appraisal  of  learning  and  understanding;  c)  instances  of  comparing  and  relating 
past  readings  or  past  experiences  to  current  readings  and  understanding;  and  d)  instances  of  self-adjusting  learning  goals 
and  objectives.  The  main  characteristics  of  reflection  here  are  evaluation,  (self)appraisal,  and  (self)adjustment. 

Comments  refer  to  any  non-interrogative  statements  concerning  readings.  Examples  of  comments  could  be:  "I 
agree  (disagree)  with...*',  "The  author  is  right  in  ...  ",  "The  question  raised  in  the  article  is  very  important The  main 
characteristic  of  comments  is  students*  voicing  judgments  or  opinions. 

Answers  refer  to  statements  which  provide  specific  information  to  the  Type  I question-information-seeking 
question.  These,  for  instance,  may  be  key  quotes,  basic  facts,  procedural  knowledge,  and  stories  from  one’s  readings. 

Discussion  and  information-sharing  notes  refer  to  general  statements  relevant  to  discussion  topics.  These  may 
include:  a)  elaboration  on  discussion  topics;  b)  exchanges  of  thoughts  or  ideas  on  related  concepts  and  issues;  c)  personal 
understanding;  and  d)  topic-related  discussing  questions. 

Scaffolding  notes  refer  to  those  providing  guidance  or  suggestion  for  discussions  or  readings.  They  can  be  notes 
from  either  the  instructor  or  the  students. 
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Table  1 . Note  Category  and  Interaction  Type 


Note  Category 

Characteristics  and  Examples 

Interaction  - 
Type 

1 

Type  I 
Question 

Ask  for  information  or  requesting  an  answer 
"What  does  hypermedia  mean?’’ 

Vertical 

2 

Typell 

Question 

Inquire,  start  a dialogue 

"How  can  we  resolve  the  control  issues  such  as 
governing  the  shared  space  when  using  a collaborative 
tool?” 

H 

3 

Answer 

Provide  answers  to  information-seeking  questions 
"Hypermedia  means  ...” 

o 

4 

Information 

sharing 

Share  information 

”My  colleague  and  I have  done  a lot  of  thinking  about 
the  nature  and  effect  of  simulations  ....  ” 

R 

5 

Discussion 

Elaborate,  exchange,  and  express  ideas  or  thoughts 
’’What  intrigues  me  from  this  week’s  readings  is  not 
how  we  define  a tool, ...  but  rather  how  tools  change 

I 

ourselves...” 

Z 

6 

Comment 

Judgmental 

”1  agree  with  A that  Schorr’s  article  was  ...” 

o 

N 

7 

Reflection 

Evaluation,  self-appraisal  of  learning 

”1  found  the  class  last  night  to  be  completely  frustrating 

T 

yet  intellectually  stimulating  . ...  it  is  what  makes  me 
think!” 

A 

8 

Scaffolding 

Provide  guidance  and  suggestions  to  others 
"...  let  us  not  move  our  lives  in  this  same  ’scripted’ 
direction.  Use  the  tool  as  an  idea  generator,  a place 
holder  of  ideas  ...” 

L 

As  indicated,  while  in  vertical  interaction,  the  individual  concentrates  on  looking  for  desired  answers  rather  than 
expressing  or  exchanging  opinions.  Whereas  in  the  horizontal  interaction,  students  express  and  exchange  views,  directly 
contributing  to  the  discussion  and  knowledge  construction. 

Constructive  learning  is  viewed  as  being  active,  cumulative,  goal-oriented,  and  constructive  (Schuell,  1988). 
However,  this  does  not  mean  that  all  constructive  learning  proceeds  or  should  proceed  according  to  all  of  these  lines  at  the 
same  time  (Simons,  1993).  In  reality,  it  may  be  that  active  learning  periods  and  passive  ones  follow  each  other, 
especially  for  learning  over  a long  period  of  time.  In  essence,  learners’  behaviors  change  in  the  process  of  learning.  They 
could  be  very  active,  highly  motivated,  and  reflective  at  one  time  but  less  active,  motivated,  and  reflective  at  another.  To 
apply  this  perspective  to  analyses  of  the  present  study,  the  discussion  participant  was  identified  as  contributor,  wanderer, 
seeker,  and  mentor  according  to  the  nature  and  the  content  of  the  notes  (See  Table  2). 


Table  2.  Participant  Category 


Participant  Category 

Involved  Categories 

Contributor 

Categories  1-8 

Wanderer 

Mainly  categories  1,4,  and  6 

Seeker 

Category  1 

Mentor 

Categories  1-8 

Each  participant  in  the  discussion  was  viewed  as  contributor  no  matter  what  type  of  note  was  contributed. 
Information-seeking  questions,  for  instance,  brought  others’  attention  to  questions  which  might  otherwise  have  been 
ignored.  The  following  quote  was  representative  of  a large  number  of  other  participant  notes  in  the  electronic  discussion. 
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"The  articles  this  week  really  made  me  think  about  the  way  people  learn,  especially  the  Envisioning  Machine 
article.  It  is  amazing  that  kids  are  able  to  go  from  not  knowing  anything  about  physics  and  eventually  are  able 
to  make  sense  of  it  without  a teacher  in  front  of  them  telling  them  what  to  do.  I think  the  key  is  that  all  of  us 
want  to  make  sense  out  of  the  world.  We  are  weird  from  birth  with  the  ability  to  take  previous  experiences  and 
build  upon  them  to  discover  new  concepts.  The  important  part  here  is  that  the  learners  did  not  do  it  on  their 
own,  but  in  order  to  make  sense  of  the  situation,  they  had  to  rely  on  each  other  and  negotiate  an  understanding." 

The  student  here  expressed  an  opinion  of  learning  and  discussed  how  children  rely  on  each  other  and  collaboratively  make 
sense  of  the  world  around  them.  By  posting  this  note,  the  student  brought  to  the  class  discussion  a unique  perspective  on 
this  learning  issue. 

Wanderers  are  the  ones  who  seem  to  be  lost,  for  at  least  the  time  being,  in  the  reading  or  the  discussion.  Those 
notes  usually  discussed  teaching  and  learning  in  general  rather  than  specific  issues  in  weekly  readings.  They  reflected  a 
specific  learning  stage  where  learners  are  floundering,  re-adjusting  themselves,  and  striving  for  an  understanding  of  the 
issue  by  relating  and  associating  different  pieces  of  information  and  knowledge.  This  stage  is  an  important  precedent  to 
learning  and  understanding.  The  wanderer’s  notes  contribute  to  the  discussion  from  a different  angle;  that  is,  not  through 
elaboration,  but  through  creating  perturbation  and  conflicts  in  the  reader. 

Seekers  are  the  ones  who  feel  an  information  deficit  and  a need  to  seek  information  in  order  to  gain  a better  or  an 
appropriate  understanding  of  the  issue.  A seeker,  for  example,  wrote:  " I don't  understand  what  they  meant  by  shared 
space.  I read  the  section  more  than  once,  but  the  idea  doesn't  want  to  sink  in  my  mind.  Can  you  help?".  The  "seeker" 
here  apparently  does  not  have  enough  knowledge  and  understanding  about  the  concept-shared  space.  Without  this,  the 
"seeker"  could  not  comprehend  the  meaning  after  reading  the  article. 

Mentors  are  those  who  when  reading  participant  notes,  try  to  understand  the  participants’  interpretation  and 
knowledge  levels  and  guide  them  in  their  reading  or  help  them  defend  and  develop  their  own  ideas  and  understanding  of 
issues.  A mentor  in  the  discussion  of  Week  12,  for  instance,  said:  " Note  A commented  that  the  IdeaFisher  could 
constrain  one’s  creative  thinking  because  you  are  using  someone  else's  opinion  of  what  things  might  be  associated  with 
other  things.  In  fact,  every  piece  of  software  you  use  could  be  considered  an  interpretive  work  at  some  level  ...  ’’. 

While  students  and  instructors  could  fall  into  any  of  these  categories  in  the  electronic  conference,  the  period  of 
time  one  is  in  the  categories  of  seeker  and  wanderer  is  usually  transitional  and  temporary. 

The  analysis  of  this  study  consisted  of  close  reading  of  every  note  and  counting  note  length  and  the  number  of 
notes  and  contributors.  Each  note  was  read  and  analyzed  at  the  sentence  and  the  paragraph  level  with  a concentration  on 
its  meaning.  Thus,  main  ideas  of  the  sentence  or  the  paragraph  were  listed  as  points  for  the  discussion.  The  pattern  of 
the  electronic  discussion  and  knowledge  construction  was  generated  using  these  main  points  and  analyzed  according  to  the 
framework  previously  discussed.  The  techniques  of  writing,  grammar,  rhetoric,  and  the  like  were  deemed  less  pertinent 
and  important  to  the  study  and  thus  were  ignored  during  the  analysis.  Likewise  foi  the  start-up,  transitional,  and  wrap-up 
words  and  phrases. 


ANALYSES 

While  a general  analysis  examined  the  semester-long  discussion,  a detailed  analysis  was  conducted  on  two 
randomly  selected  weeks  to  examine  the  pattern  of  electronic  discussions  and  knowledge  construction,  the  role  of  the 
instructor  and  the  student,  and  ways  of  mentoring  used  in  the  electronic  conference.  The  weeks  selected  and  their  topics 
were  Week  9--"Science  tools  for  collaboration  in  a learning  community"  and  Week  1 2-"Music,  art,  visualization  & 
animation  tools  for  creativity  & critical  thinking". 
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Table  3.  General  Analysis  for  a 15  Week  Discussion 


Week 

Topic 

Number 
of  notes 

Number 

of 

contrib- 

utors 

Average 

line 

length 

Instructors' 
notes  & 
percentage 

1 

Introduction:  Trends  in  computers  as 
cognitive  and  sociomedia  tools 

10 

7 

7.68 

5 (50%) 

2 

Linking  tools  to  learner  centered 
psychological  principles 

17 

13 

20.5 

3 (17%) 

3 

Learner  centered  design 

16 

14 

40.9 

6 (37%) 

4 

Multimedia  composition  and 
knowledge  construction 

35 

18 

41.06 

6 (17%) 

5 

Writing  tools  for  idea  generation  and 
cognitive  enhancement 

24 

16 

54.6 

5 (24%) 

6 

Distance  writing  collaboration  tools 
& computer-mediated  communication 

24 

16 

32.14 

7 (29%) 

7 

Internet,  Mosaic,  WWW,  & other 
information  systems 

29 

16 

30.2 

6 (20%) 

8 

Science  tools  for  collaboration  in 
learning  community 

40 

18 

30.8 

4 (10%) 

9 

Science  tools  for  conducting  inquiry 

32 

15 

29.8 

3 (9%) 

10 

Math  tools  for  problem  solving  and 
problem  representation 

31 

15 

35.35 

10  (31%) 

11 

Computer  programming  & CAD 
systems  for  designing  knowledge 

32 

19 

39.78 

7 (21%) 

12 

Music,  art,  visualization  & animation 
tools  for  creativity  & critical  thinking 

23 

17 

36.4 

2 (9%) 

13 

Virtual  reality,  intelligent  tools,  & 
other  dreams  of  reality 

32 

19 

37.09 

6 (18%) 

14 

Student  self  selection  Week 

21 

18 

28.5 

2 (9%) 

15 

Cognitive  and  sociomedia  tools 
revisited 

14 

12 

44 

1 (7%) 

* 

Other  creative  reflections 

8 

8 

18.75 

0 

** 

Tool  taxonomy  presentation  (one  of 
the  assignments) 

20 

12 

16.35 

8 (40%) 

Total 
/Aver- 
in  

408/ 

27 

14 

32  lines 

* "Other  creative  reflections"  is  not  a weekly  discussion  topic,  but  notes  contributed  by  students  throughout  the  semester 
about  their  reflective  thoughts. 

**  Tool  taxonomy  presentation  is  one  of  the  class  assignments.  Students  put  their  discussion  notes  about  the  assignment 
under  this  topic  exclusively. 
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General  analysis 

Quality.  Fifteen  weeks1  electronic  discussions  generated  a total  number  of  408  notes  (See  Table  3)  from  1 7 
discussion  topics.  The  average  number  of  notes  contributed  each  week  was  27  . The  note  length  ranged  from  8 lines  in 
the  first  week  to  55  lines  in  the  fifth  week.  The  average  note  length  across  the  17  topics  was  approximately  32  lines. 
Each  line  was  roughly  10  words,  making  an  average  note  length  of  approximately  320  words.  Of  all  the  discussions  in 
15  weeks,  only  the  first  week  had  an  average  length  of  eight  lines.  The  rest  all  exceeded  an  average  of  20  lines.  Three 
hundred  and  twenty  words  could  fill  one  page  and  550  words  could  be  printed  on  almost  two  pages  double  spaced.  The 
quality  of  the  electronic  conference  is  closely,  though  relatively,  related  to  its  quantity  or  note  length.  Clearly,  students 
here  were  not  simply  raising  information-seeking  questions,  but  were  really  writing  thoughtful  reflections,  discussions, 
and  comments. 

Students*  involvement.  The  number  of  contributors  was  steady  across  all  the  discussion  topics  and  in 
appropriate  proportion  with  the  number  of  notes  in  each  week  given  the  number  of  student?,  taking  the  class.  The 
smallest  number  of  contributors  was  7 and  the  largest  was  19  in  the  class  of  twenty-two  students.  Over  73%  of  the 
students  participated  in  the  electronic  discussion  each  week,  with  a participation  rate  ranging  from  35%  in  the  first  week 
and  95%  in  the  eleventh  week.  If  we  excluded  the  first  week,  when  students  were  starting  to  use  the  electronic 
distribution  list  for  this  class,  the  participation  rate  would  be  even  higher.  Clearly,  the  discussion  space  was  not 
dominated  by  a few  students,  nor  was  it  swamped  with  socializing  statements,  greetings,  or  information-seeking 
questions  which  might  be  used  to  earn  a participation  grade  for  this  class.  As  indicated  above,  students  mostly  posted 
thought-provoking  questions,  discussions,  and  reflections. 

Instructors'  involvement.  Of  the  total  number  of  notes,  86  notes  were  contributed  by  the  instructors  and 
accounted  for  an  average  of  20%.  The  percentage  ranged  from  7%  in  Week  15  to  50%  in  Week  1.  Scaffolding  was  high 
at  the  beginning  of  the  semester.  For  instance,  in  the  first  week,  the  instructors'  notes  accounted  for  half  of  the  class 
notes.  Most  of  these  notes  were  to  suggest  discussion  parameters,  call  for  students'  participation,  and  ensure  that  the  E- 
mail  distribution  list  worked  and  students  responded  to  it.  As  the  semester  moved  on,  instructors'  notes  gradually  reduced 
to  around  20%  with  the  exception  of  Week  3 and  Week  6 in  which  the  instructors'  notes  reached  over  30%.  The 
discussion  topic  of  Week  3-leamer-centered  design-drew  interest  from  both  students  and  instructors.  The  discussion 
topic  in  Week  6-writing  tools  for  idea  generation  and  cognitive  enhancement-was  one  of  the  instructors'  special  interest 
area,  where  he  had  vast  research  experiences.  In  that  week's  discussion,  he  shared  his  expertise  and  guided  students  in 
understanding  the  use  of  computer  tools  to  help  generate  ideas  and  enhance  cognitive  development. 


Specific  analysis 

Starter  Notes.  Detailed  data  analyses  focused  on  starter,  wrapper,  instructor,  and  student  participant  notes,  on 

specific  discussion  topics,  and  any  other  notes  posted  in  Week  9 and  Week  12*.  For  our  analyses,  main  ideas  of  each 
note  were  summarized,  synthesized,  and  listed  in  tables.  Each  note  reflected  each  individual's  unique  understanding  and 
represented  individual's  unique  contribution  to  the  discussion.  The  starter’s  note,  to  some  extent,  led  a week’s  discussion. 
For  example,  the  starter  of  Week  9 wrote: 

" I think  one  of  the  issues  we  need  to  address  is  relevancy.  I found  it  a lot  easier  to  appreciate  the  role  of 
Sherlock  I (Lajoie's  article)  and  the  Medical  Center  program  (Anderson's  article)  as  an  instructional  aid,  than  I did 
when  I read  about  Bio-World  (Lajoie's  article)  or  the  Envisioning  Machine  (Teasley's  article).  Both  Sherlock  I 
and  Medical  World  have  a high  degree  of  relevance  to  their  users,  who  desire  to  benefit  from  using  these 


* Assigned  articles  for  Week  9 are:*  1 ) Constructing  a joint  problem  space:  the  computer  as  a tool  for  sharing  knowledge;  2) 
Computer  environment  as  cognitive  tools  for  enhancing  learning;  3)  Medical  center:  a module  hypermedia  approach  to 
problem  design;  4)Video  labs:  tools  for  scientific  investigation;  and  5)  Wireless  coyote:  a computer-supported  field  trip. 

Assigned  articles  for  Week  12  are:  1)  Socratic  approach  to  use  computers  with  at-risk  students;  2)  Visual  languages  for 
cooperation:  a performing  medium  approach  to  systems  for  cooperative  work;  3)  Prototyping  multimedia:  lessons  from  the 
visual  computing  group  at  Project  Athena  Center  for  Educational  Computing  Initiative;  and  4)  Programs  that  help  you  think 
creatively  and  plan  effectively. 
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programs.  Granted  the  Medical  Center  program  and  the  Envisioning  Machine  can  promote  problem  solving,  but 
is  problem  solving  useful  or  always  possible  in  the  absence  of  relevancy  to  the  user?" 

The,  starter  here  raised  and  discussed  the  issue  of  relevancy  in  using  tools.  The  issue  was  then  picked  up,  discussed,  and 
commented  on  by  six  other  individuals  during  Week  9 (See  Table  4). 


Table  4.  Discussion  on  the  Issue  of  Relevancy  in  Using  Tools  during  Week  9 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

c 

N.  1** 

Discuss  the  issue  of  relevancy  in  using  tools 

Discussion 

c 

N.  5 

Discuss  relevancy  issue  raised  in  Note  1 

Discussion 

c 

N.  6 

Discuss  relevancy  issue  (Note  1) 

Discussion 

c 

N.  7 

Comment  on  relevancy  in  applying  tools 

Comment 

c 

N.  13 

Discuss  the  relevancy  issue  (citing  three  comments  from 
the  previous  notes)  and  suggest  that  relevancy  is  related  to 
knowledge  transfer,  far  and  near. 

Discussion 

c 

N.  15 

Reflect  on  the  issue  of  relevancy 

Reflection 

(C=  contributor;  W=  wanderer;  S=  seeker;  and  M=  mentor) 


In  both  Week  9 and  Week  12,  students  tended  to  carefully  read  the  starter's  note  and  answer  the  questions  or 
discuss  issues  raised.  The  starter  of  Week  12  discussed  visuals  used  in  daily  life  and  education,  and  raised  a number  of 
discussing  questions  about  weekly  readings.  He  wrote;  "The  art  or  visual  has  been  disregarded  in  education  for  a long 
time.  Now,  we  recognize  that  the  information  age  is  multidimensional  and  the  computer  is  a way  for  us  to  grasp  the 
enormous  amount  of  usable  information.  We  see  computers  as  new  ways  of  seeing.  Some  questions  rise  from  here: 
How  do  we  conceptualize  with  vision?  How  does  the  use  of  computational  visual  affect  learner's  cognitive 
development?".  Once  again,  the  starter's  note  initiated  a heated  discussion  on  visualization  and  learning  in  Week  12  (See 
Table  5). 


Table  d.  uiscuss 

Participant 

Category 

ion  on  v lsua 
Note 

lizauon  ana  i_-cauuiiK  uuiiiik  wcck 

Main  Ideas 

Note 

Category 

C 

N.  1 

Discuss  the  number  of  visuals  used  in  daily  life  and  old 
saying  —picture  is  worthy  a thousand  words 
Discuss  that  visuals  have  been  neglected  in  education 
Raise  inquiring  questions  (1-5) 

Discussion 
Type  II  Qs 

C/M 

N.  5 

Comment  on  the  blackboards  in  the  halls  of  the  Chemistry 
building  and  lack  of  it  in  Education  building 
Share  experience  with  the  class  on  visualization 

Comment 

Information 

Sharing 

C 

N.  6 

Discuss  Latin's  article  (creativity  - people  use  each  other's 
ideas  to  develop  a better  end  product)  and  raise  questions 
concerning  working  with  group 
Reflect  on  personal  experience  working  in  groups 

Discussion 

Reflection 

C/M 

N.  7 

Discuss  Vmacs 

Discussion 

**  The  note  number  signifies  the  order  in  which  the  note  was  contributed.  The  number  uniquely  identified  each  note  but  does 
not  carry  any  other  meaning. 
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Table  5.  (continued) 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

c 

N.  10 

Discuss  visualization 

Discussion 

C 

N.  18 

Summarize  different  comments  on  visualization 

Discussion 

C/ M 

N.  19 

Response  to  one  previous  question— visualization  seems  to 
be  rising  in  importance 

Answer 

Scaffolding 

C/M 

N.  22 

Share  experience  on  how  teachers  and  students  collaborate 
in  teaching  and  learning 
Discuss  visualization 

Discussion 
Type  II  Qs 

Table  5 lists  notes  from  eight  students  during  Week  12.  These  notes,  though  falling  into  different  categories 
such  as  comment,  discussion,  information-sharing,  and  scaffolding,  all  centered  around  the  discussion  of  how 
visualization  being  used  in  daily  life  as  well  as  in  instruction.  Quite  a few  referred  to  the  blackboards  in  the  chemistry 
building  on  campus  and  commented  that  the  blackboard  served  as  a space  for  visualization  and  for  exchanging  ideas  as 
well.  One  mentor  here  related  visualization  to  tools  and  learning,  and  reiterated  the  importance  of  visualization  in 
learning.  Mentors  acquainted  peers  with  some  real  world  practices  for  using  visualization  in  teaching  and  learning. 

Wrapper  Notes.  The  wrap-up  for  the  weekly  discussion  did  not  demonstrate  its  expected  value  to  synthesize  the 
groups'  understanding  of  the  readings.  In  most  weeks,  the  wrapper  simply  expressed  his/her  personal  feelings  and 
summarized  the  individual  understanding  of  topics  discussed.  For  example,  the  wrapper  for  Week  9 wrote:  "I  particularly 
enjoyed  the  readings  for  this  week.  I am  sure  that  part  of  the  reason  is  that  they  were  concerned  with  subject  matter  that  I 
am  familiar  with,  so  my  ability  to  envision  the  tools  discussed  was  enhanced. ..  I agree  with  student  B that  the  thrust  of 
Lajoie's  article  are  the  4 tools  that  assist  learners  to  accomplish  cognitive  tasks."  In  some  other  weeks,  the  wrapper 
technically  summarized  what  each  student  and  instructor  contributed  without  pulling  them  together  or  synthesizing  these 
ideas.  For  example,  the  wrapper  for  Week  12  wrote:"...  Student  C raised  a question  of  'the  chalk  board  in  the  Chemistry 
Building’  which  a couple  of  people  replied  that  they  will  agree  with  this  idea-gathering  information  from  more  than  one 
source...".  Wrappers  read  discussion  notes,  reflected  on  them,  but  often  offered  few  insights  or  summaries. 

Instructor  Notes.  During  Week  9,  the  instructors  contributed  three  notes  which  accounted  for  9%  of  the  total. 
Three  notes  involved  general  comments  to  the  class  and  suggestions  for  future  learning.  The  instructor  reflected  on  the 
history  of  computing  and  tools  evolution,  commented  on  tools  and  theories  guiding  the  development  of  tools,  and 
encouraged  students  to  take  part  in  the  role  play  (See  Table  8).  He  discussed  different  types  of  tools  related  to  five  levels 
of  learning  as  well  as  four  different  types  of  cognitive  tools  identified  in  the  readings.  Additionally,  he  pointed  to  his 
own  experiences  to  guide  students  in  understanding  the  concept  of  cognitive  tools.  For  instance,  he  wrote:  "She  (Lajoie) 
says  that  there  are  4 types  of  cognitive  tools  that  can  be  identified  by  the  functions  that  they  serve...  they  support  my 
memory  and  strategic  thinking,  give  me  the  facts  so  I don’t  have  to  waste  my  time,  enable  me  to  engage  in  new 
activities,  and  get  me  to  test  and  explore  new  things...".  Such  restatement  and  simplification  of  the  readings  were 
typically  offered  by  the  instructor  after  the  weekly  starter's  comment  and  students'  wandering  or  queries.  According  to  the 
instructor,  this  was  useful  and  necessary. 


831 


Table  6.  The  Instructor’s  Notes  during  Week  9 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

C/M 

N.  9 

Reflect  on  the  history  of  computing, 
Reflect  on  tools  evolution. 
Encourage  debate  and  role  play 

Reflection 

Suggestion 

C/M 

N.  14 

Comment  on  surfing  the  net  and  the  related  jobs. 
Comment  on  tools  vs.  theories 

Comment 

Scaffolding 

C/M 

N.  24 

Comment  on  learning  in  the  class, 

Share  information  about  other  available  materials  related 
to  the  class. 

Suggest  students*  writing  proposals  for  conferences. 

Comment  on  the  previous  class  (agree  with  Note  22), 

Discuss  different  kinds  of  tools  and  levels  of  learning. 

Discuss  four  types  of  cognitive  tools  identified  in  one  reading 

Scaffolding 

Comment 

Suggestion 

Discussion 

Information 

Sharing 

During  Week  12,  the  instructor  contributed  two  notes,  comprising  9%  of  the  total.  The  instructor  commented 
on  the  metaphor  used  by  one  student  to  describe  thinking  and  clarified  that  we  are  emphasizing  process  skills  in  education 
today.  He  challenged  students  with  such  questions  as  whether  computers  really  can  teach  such  process  skills.  Next,  the 
instructor  responded  to  a participant  note  about  how  computer  software  can  assist  cognitive  development.  He  further 
illustrated  how  a tool,  e.g.  Writing  Partner,  can  help  people  think  and  plan  creatively.  Using  his  own  experience  and 
understanding,  the  instructor  guided  students'  learning  by  reading  their  notes  and  explaining  how  tools  can  be  used  to  help 
people  think  and  plan.  In  reflection,  the  instructor  later  said  that  he  was  always  focusing  on  assisting  students  in  their 
understanding  of  the  issue. 

Topic  Discussions  for  Week  9.  The  topic  of  collaboration  brought  together  five  students  to  exchange  ideas  and 
opinions.  They  engaged  in  a lively  debate  about  differences  between  cooperation  and  collaboration  and  examined  the 
relationship  between  the  information  explosion  and  collaboration.  One  student  wrote:  "They  (authors)  make  a distinction 
between  collaborative  and  cooperative  problem  solving  that  I hadn't  given  much  prior  thought  to.  Cooperative  meant 
that  there  was  a division  of  labor  in  the  problem-solving  but  collaborative  meant  that  everyone  was  engaged  in  trying  to 
find  a solution."  Another  student  related  the  information  explosion  to  collaborative  learners  and  workers:  "Because  of 
knowledge  explosion, ...  they  (students)  must  learn  to  be  collaborators,  because  being  able  to  access  the  information 
doesn't  mean  knowing  how  to  apply  it  or  if  it  is  the  right  information  to  apply." 

The  discussion  in  Week  9 well  illustrated  Dewey's  point  that  reflection  is  a way  of  learning.  Through  thirteen 
notes  in  Week  9 (See  Table  7)  the  students  engaged  in  reflective  activities  on  the  class,  its  readings,  ways  of  learning,  and 
sharing  information,  and  comments  on  the  role  play  activity. 

Table  7.  Reflective  Thoughts  in  the  Discussion  during  Week  9 


Participant 

Category 

Note  ' 

Main  Ideas 

Note 

Category 

C 

N.  7 

Comment  on  learning  in  general 

Reflection 

C 

N.  11 

Discuss  the  reading  while  playing  a role 

Reflection 

C/M 

N.  14 

Comment  on  surfing  the  net  and  the  related  jobs 

Reflection 

c 

N.  15 

Comment  — can't  play  a role 

Reflection 

C/M 

N.  9 

Encourage  debate  and  role  play 

Suggestion 

s 
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Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

c 

N.  19 

Reflect  on  readings  in  general 

Reflection 

s 

N.  20 

Question  about  technical  problems  in  VaxNotes 

Type  I Q. 

c 

N.  23 

Reflect  on  the  class  (completely  frustrating  yet 
intellectually 
stimulating) 

Comment  on  the  instructors’  summaries  and  comments 
made  in  the  previous  class 

Comment  on  the  week’s  reading  (enjoy  because  of  the 
content  familiarity) 

Reflection 

Comment 

Comment 

C/M 

N.  24 

Comment  on  learning  in  the  class 

Comment  on  the  previous  class  (agree  with  Note  22) 

Reflection 

C 

N.  25 

Reflect  on  the  week’s  reading  and  learning 

Reflection 

C 

N.  28 

Share  experience  of  young  children  learning  computer 
games 

Share  experience  in  high  school  science  class  and  comment 
on  the  science  class  with  Education  technology  tools 
(would 

enioy  more  if  tools  were  available) 

Reflection 

Discussion 

C 

N.  12 

Share  information  on  a job  related  to  Education  technology 

Information 

Sharing 

C 

N.  16 

Reflect  on  the  class  and  what  has  been  learned 

Reflection 

Almost  all  of  these  reflections  on  learning  contributed  to  the  discussion,  and  some  students  acted  as  mentors  at 
the  same  time.  Further,  they  played  assigned  roles  (e.g.  optimist,  watchdog,  speculator,  and  questioner).  Role  play 
activities  encouraged  other  students  to  be  actively  involved  in  the  discussion. 

Discussion  of  general  learning  and  thinking  skills  was  another  common  topic  during  Week  9 (See  Table  8), 
when  about  ten  individuals  discussed  these  and  other  related  issues. 


Table  8.  Discussion  on  General  Learning  and  Thinking  Skills  during  Week  9 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

C 

N.  1 

Raise  questions  about  the  validity  of  two  studies  in  weekly 
readings 

Type  HQ 

C/M 

N.  9 

Reflect  on  the  history  of  computing 

Reflection 

C/W 

N.  13 

Comment  on  structured  learning  vs.  unstructured  learning 

Comment 

C/M 

N.  14 

Comment  on  tools  vs.  theories 

Comment 

C/W 

N.  16 

Discuss  constructivism  and  tools 
Raise  questions  about  constructivism 

Discussion 
Type  II Q. 

C/W 

N.  22 

Discuss  knowledge  transfer 

Discussion 

833 
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Table  8.  (continued) 


Participant 

Category 

Note 

Main  deas 

Note 

Category 

c 

N.  25 

Discuss  student  learning,  problem  solving,  and  critical 
thinking  skills 

Discussion 

c 

N.  26 

Discuss  and  raise  questions  about  the  nature  of  learning 

Discussion 

c 

N.  28 

Discuss  floundering  in  learning  (through  playing  computer 
games) 

Discussion 

c 

N.  27 

Comment  on  problem-solver  (reply  to  the  previous  note) 

Comment 

They  interacted  as  contributors,  mentors,  and  sometimes  wanderers.  One  of  the  mentors  reviewed  the  history  of 
computing,  providing  some  background  knowledge  for  a better  understanding  of  how  tools  evolved  to  assist  learning. 

The  mentor  also  highlighted  the  relation  and  nature  of  tools  and  theories,  thus  supporting  students  in  their  understanding 
of  how  theories  are  being  used  to  guide  the  design  of  tools.  Wanderers  contributed  to  the  discussion  through  the  analysis 
of  structured  and  unstructured  learning,  constructivism,  and  tools.  They  wondered  whether  these  issues  related  to 
collaboration,  virtual  field  trips,  and  others.  If  they  were  interconnected,  the  wanderers  had  yet  to  discover  these 
associations.  Wandering  did  not  hinder  their  learning.  In  fact  wandering  helped  others  learn,  because  the  wanderers'  notes 
added  fuel  to  the  discussion,  created  cognitive  conflicts,  and  urged  the  reader  to  think.  As  Piaget  would  contend,  their 
queries  caused  other  participants  to  experience  disequilibrium  and  cognitive  dissonance  which  needed  to  be  resolved 
through  additional  reflection,  discussion,  and  negotiation. 

The  following  paragraph  is  a good  example  of  a wandering  note  in  Week  9.  It  said:  "Transfer  is  important  and 
depth  is  important.  Students  have  to  have  opportunities  to  explore  a concept  in  depth  and  make  transfer  from  that  depth. 

I think  it's  the  teacher’s  job  to  stir  that  curiosity  and  that's  when  the  learning  becomes  relevant,  when  from  a 
constructionist's  viewpoint,  you  start  from  the  child’s  prior  knowledge.  From  there  you  guide  them  to  exploration  and 
transfer  and  application."  This  note  discussed  very  generally  learning,  critical  thinking  skills,  and  knowledge  transfer 
without  giving  or  providing  any  specific  references  to  weekly  readings,  but  instead,  it  drew  heavily  from  experiences. 

This  note  added  much  food  for  thought  in  participant  discussions. 

Topic  Discussions  for  Week  12.  Several  major  topics  of  discussion  emerged  from  the  discussion  of  Week  12. 
They  were  reflective  activities,  the  issue  of  using  computers  with  at-risk  students,  and  the  issue  of  software  programs  that 
help  people  think  and  plan.  Four  individuals  in  Week  12  reflected  on  this  distance  learning  course  and  the  weekly 
readings.  For  example,  one  student  wrote:  "Once  again,  I’m  excited  about  the  readings  for  this  week.  In  the  beginning  of 
the  semester  I was  intrigued  with  the  technology  of  education  and  was  at  a point  of  believing  that  using  technology  could 
save  our  educational  system.  I really  am  understanding  that  technology  alone  really  has  no  bearing  on  how  someone  is 
able  to  learn  better,  it's  a very  complicated  process  that  I’ve  become  more  aware  of..."  Another  student  wrote:  "I  would 
like  to  say  that  I am  REALLY  getting  the  feeling  of  this  distance  thing.  As  you  may  not  be  aware,  I am  in  Syracuse, 
New  York  for  the  rest  of  the  semester ....  I won’t  be  in  class,  but  I will  be  participating  in  the  class  discussion  in 
VaxNotes.  I can’t  think  of  a better  class  where  this  situation  would  be  more  suitable."  (This  student  was  in  the  hospital 
during  the  last  month  of  the  semester.)  Both  students  revealed  their  true  feelings  towards  the  course.  They  started  from 
totally  divergent  routes  but  came  to  a converging  point.  That  is,  they  felt  they  had  learned  a lot  from  reading  and 
participating  in  the  weekly  electronic  discussion. 


Table  9.  Discussion  on  the  Use  of  Compiler  Tools  with  At-Risk  Students  During  Week  12 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

c 

N.  1 

Discuss  Pogrow's  article  and  raise  3 discussing  questions 
about  at  risk  students 

Discussion 
Type  II  Os 

C/M 

N.  2 

Discuss  Pogrow’s  article  and  argue  that  the  issue  is 
teaching  problem-  solving  skill  in  general 

Discussion 

c 

N.  3 

Share  colleague’s  opinion  on  simulation 

Information 

Sharing 

Q s V 
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Table  9.  (continued) 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

C/M 

N.  5 

Discuss  Pogrow's  article,  agreeing  with  most  ideas  except 
"software  as  instructional  objects... 

Discussion 

C 

N.  6 

Discuss  Pogrow's  article  and  the  approach  (both  agree  and 
disagree) 

Discussion 

C/M 

N.  7 

Discuss  Pogrow's  article  — drama  heightens  curiosity  and 
motivation  and  promotes  the  emotional  engagement  that 
deepens  learning. 

Discussion 

c 

N.  8 

Comment  on  Pogrow's  article  and  agree  with  the  starter 
Disagree  with  the  author  on  at-risk  students  learning  issue 

Comment 

Discussion 

C/M 

N.  9 

Agree  with  one  previous  note  — "general  problem  solving 
skills" 

Comment 

C/M 

N.  12 

Share  experiences  working  with  at-risk  students 
Answer  one  question  from  the  starter  - “when  do  we  let  at- 
risk  students  back  into  the  normal  classroom?" 

Information 

Sharing 

Answer 

c 

N.  16 

Discuss  Pogrow’s  article  - 1)  how  instructors  use  software; 
2)  software  and  tools  are  not  answers  to  education 
problems;  3)  creative  use  of  tools 

Discussion 

c 

N.  17 

Discuss  the  readings  - drama,  visualization,  creativity 

Comment 

c 

N.  18 

Summarize  different  comments  on  Pogrow's  article 

Discussion 

c 

N.  20 

Discuss  Pogrow  - "sophistication  of  the  learning  produced 
by  technology,  depends  on  the  sophistication  of  the 
conversation  surrounding  its  use,  not  the  sophistication  of 
the  technology." 

Discuss  Pogrow's  emphasis  on  meta-cognition 

Discussion 

C/M 

N.  22 

Discuss  meta-cognition 

Discuss  at-risk  students  - "at  some  time/different  time  we 
are  all  "at  risk". 

Discussion 
Type  II  Qs 

During  Week  12,  fourteen  individuals  engaged  in  a lively  sub-discussion  about  computers  being  used  with  at- 
risk  students.  They  shared  information  and  personal  experiences  regarding  working  with  at-risk  students,  and  discussed 
the  author's  approach  to  at-risk  students.  Some  answered  an  information  seeking  question.  Some  analyzed  the  nature  of 
software  used  with  at-risk  students  and  problems  with  at-risk  students.  Others  challenged  the  author's  approach  to  at-risk 
students  and  the  class  by  saying  that "...  at  some  time/different  time  we  are  all  'at  risk'.''  Finally,  some  mentors 
suggested  that  other  students  should  think  about  teaching  general  problem-solving  skills  with  at-risk  students  because 
problem-solving  skills  are  essential  to  all  learners. 
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Tabic  10.  Discussion  on  Software  That  Helps  People  Think  Creatively  and  Plan  Effectively  during  Week  12 


Participant 

Category 

Note 

Main  Ideas 

Note 

Category 

C 

N.  1 

Discuss  Schorr’s  article  and  raise  4 questions 

Discussion 
Type  II  Os 

C/M 

N.  2 

Discuss  Schorr's  article,  commenting  that  IdeaFisher  would 
limit  creativity  more  than  it  could  increase  it. ... 
Inspiration  allows  for  more  creativity... 

Discussion 

c 

N.  3 

Discuss  IdeaFisher  and  argue  that  every  piece  of  software 
could  be  an  interpretive  work  at  some  level. 

Discussion 

c 

N.  6 

Provide  answer  to  one  of  the  starter’s  question  — ”do  we 
need  a computer  to  develop  these  types  of  thinking 
strategies?” 

Discussion 

Answer 

c 

N.  8 

Comment  on  Schorr’s  article,  enjoying  Inspiration 

Comment 

C/M 

N.  9 

Discuss  Schorr’s  article  from  special  education  point  of 
view 

Agree  with  one  previous  note  — "the  effect  of  the  IdeaFisher 
lies  in  the  inspiration  created  by  effective  brainstorming” 

Discussion 

Comment 

c 

N.  16 

Discuss  Schorr’s  article,  Inspiration,  and  Widget 

Discussion 

c 

N.  17 

Discuss  Schorr's  article  and  agree  with  the  starter  (article 
carries  a sale  pitch)  but  personally  like  software  help 
create  ideas 

Comment 

C/M 

N.  19 

Discuss  Writing  Partner 

Discussion 

Scaffolding 

About  10  students  in  Week  12  engaged  in  a discussion  of  tools  that  help  people  think  creatively  and  plan 
effectively.  The  discussion  revealed  individual  opinions  as  well  as  a common  thread-how  can  tools  help  people  think 
creatively  and  plan.  For  example,  one  student  wrote:  " I thought  that  IdeaFisher  would  limit  creativity  more  than 
increase  it.  Doesn’t  this  tend  to  destroy  our  own  creativity  and  turn  us  into  lazy  couch  potatoes?”.  Another  student 
voiced  the  opposite  opinion:  "You  use  this  word  (here  it  refers  to  ‘potato’)  as  a springboard  to  idea  generation.  This 
method  sounds  similar  to  the  word  list  in  IdeaFisher.  It  really  doesn’t  limit  you  in  terms  of  creative  thinking  any  less 
than  brainstorming.  In  fact,  it  opens  you  up  because  it  forces  you  to  find  a way  to  use  a word  like  ’potato’  to  generate 
ideas  about  a problem  you’d  like  to  solve." 

One  student  said:  ”1  thought  that  Inspiration  allowed  for  more  creativity  since  the  tool  did  not  prod  or  lead  the 
learner."  Another  student  echoed:  ”1  believe  that  everyone  is  in  need  of  such  a program-inspiration.  Everybody  would 
like  to  find  some  sort  of  help  when  it  comes  to  arranging  ideas.  The  visual  presentation  of  one’s  thoughts,  rearranging 
them  in  order  to  create  his/her  own  ideas,  changing  relationships  and  so  forth  makes  this  tool  indeed  impressive  in  my 
view.” 

Opinions  on  computer  programs  that  help  people  think  creatively  and  plan  effectively  were  quite  diverse.  The 
notes  demonstrated  personal  preferences  for  certain  programs  and  also  revealed  both  excitement  and  doubts  about  tools 
that  can  help  people  think  and  plan.  To  further  assist  students’  discussion  about  tools  and  understanding  of  how  they 
worked,  some  of  these  tools  were  later  demonstrated  in  class. 

Summary  of  Discussions  in  Week  9 and  12.  The  analysis  showed  that  four  major  discussion  topics  emerged 
from  the  readings  and  were  discussed  at  length  during  Week  9.  They  were:  1 ) general  discussion  on  the  nature  of 
learning,  critical  thinking  skills,  problem  solving  skills,  etc.;  2)  discussion  of  tools  for  collaborative  learning;  3)  the 
issue  of  relevancy  in  the  use  of  computers;  and  4)  rejections  on  the  class  and  weekly  readings.  In  Week  12,  four  major 
themes  or  threads  ran  through  the  discussion:  1)  general  discussion  on  thinking  skills  and  technological  tools  for 
learning;  2)  reflections  on  the  class,  readings  and  learning;  3)  discussion  of  tools,  such  as  IdeaFisher,  Inspiration,  and 
Writing  Partner  that  help  people  think  creatively  and  plan  effectively;  and  4)  discussion  of  tools  and  their  uses  in  special 
education. 
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The  analysis  of  two  weeks'  discussions  revealed  that  almost  all  students  were  contributors  and  the  instructors 
were  mentors  during  electronic  discussions.  The  notes  were  mostly  in  the  categories  of  discussion,  comment,  reflection, 
information  sharing,  and  scaffolding  (See  Table  1 1).  There  were  only  two  type  I questions  (e.g.  What  does  it  mean  by 
“share  space"?).  The  interaction  type  during  the  two  weeks’  discussions  was  predominantly  horizontal. 

Table  1 1 . Summary  of  Note  Category,  Participate  Category,  and  Interaction  Type  in  Week  9 and  Week  1 2 


Note/Participation  Category  * 

Week  9 

Week  12 

Interaction  Type 

Type  I question 

1 

1 

Vertical 

Type  II  question 

4 

10 

H 

Discussion 

23 

30 

o 

Answer 

0 

4 

R 

Information  sharing 

2 

4 

I 

Comment 

16 

14 

Z 

Reflection 

6 

2 

o 

Scaffolding 

4 

8 

N 

Contributor 

46 

43 

T 

Wanderer 

1 

4 

A 

Mentor 

22 

8 

L 

Seeker 

0 

1 

Vertical 

RESULTS  AND  DISCUSSION 


Patterns  of  Electronic  Discussion 

The  analysis  has  shown  some  interesting  patterns  in  the  discussion.  First,  the  electronic  discussion  usually 
centered  around  some  major  themes  emerging  from  the  weekly  readings.  For  example,  in  Week  9,  six  notes  discussed 
cognitive  tools  and  the  relevancy  of  tools  used  in  learning.  However,  the  analysis  has  also  shown  that  having  one 
discussion  topic  covered  by  a group  of  students'  notes  does  not  eliminate  the  divergence  and  diversity  of  the  discussion. 
That  is,  though  each  note  was  discussing  the  same  topic,  each  perspective  was  quite  idiosyncratic  and  bared  on  the 
individual’s  understanding  and  experiences.  To  illustrate  this  point,  consider  Note  5 and  Note  6 in  Week  9.  Note  5 was 
responding  to  Note  1 and  discussed  the  issue  of  tools  relevancy  and  so  was  Note  6.  However,  Note  5 wrote: 

“Anderson's  article  on  Medical  Center  gives  us  the  notion  of  how  learning  is  facilitated  by  higher  order  of 
thinking.  The  issue  that  Student  A brought  forward  regarding  the  concept  of  transferability  of  information  and 
relevancy  is  good.  In  order  for  the  software  to  be  relevant,  it  has  to  set  goals.  One  of  the  goals  in  designing 
Medical  Center  is  to  give  learners  the  opportunity  to  build  a schema  and  relate  content  knowledge  to  its 
application  in  clinical  problem  solving." 

Note  6 wrote: 

“I  think  relevancy  is  what  the  teacher's  role  is  and  also  how  teachers  make  the  specific  tools  relevant  to  their 
course  and  learning  in  general.  The  process  of  problem  solving  is  a very  important  goal  in  any  course  (just 
look  at  the  Indiana  standards  for  public  education).  I admit  the  context  that  the  Medical  Center  program  and 
the  Envisioning  Machine  is  not  relevant  to  every  subject,  but  the  process  of  problem  solving  that  these 
individual  subjects  teach  can  be  applied  to  all  areas." 


* Most  notes  comprised  of  several  paragraphs  and  each  paragraph  was  usuallv  devoted  to  the  discussion,  comment,  and 
reflection  of  one  idea.  Thus,  a note  can  fall  into  several  categories. 
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The  convergence  on  the  major  discussion  themes  did  not  limit  the  depth  of  discussion  or  the  understanding;  on  the 
contrary,  it  brought  multiple  perspectives  to  the  topic  under  discussion,  thereby  providing  an  extremely  rich  context  for 
students  to  construct  their  individual  understanding  of  the  issue. 

Secondly,  this  electronic  media  supported  both  the  vertical  and  the  horizontal  interactions.  The  participants  were 
at  some  time  equal  peers,  but  other  times  a few  peers  acted  as  guides  or  mentors  for  other  students'  learning.  For 
example,  one  student  in  Week  6 wrote: 

"We  all  seem  to  be  concerned  with  what  technology  will  *do*  to  us.  Will  it  improve  our  thought  processes? 
Will  it  stunt  our  creativity?  Will  it  augment  or  hinder  our  collaboration?  I think  there's  a danger  in  giving  so 
much  agency  to  technology.  We  have  to  remember  where  tools  come  from.  They  come  from  our  minds.  So, 
tools  might  be  catalysts  for  improvement  of  our  thought  processes,  but  the  agent  of  change  is  ourselves." 

This  student's  note  reminded  the  class  of  a very  important  idea  that  tools  should  be  extensions  of  ourselves,  not 
replacements  of  ourselves  or  agents  to  direct  our  relationships  to  learners.  We  should  bear  this  in  mind  when  we 
incorporate  tools  into  instruction.  Notes  like  this  may  have  contributed  more  to  the  discussion  due  to  their  meaning  and 
importance.  Accordingly,  these  students  contributed  more  to  that  particular  discussion  topic  than  to  others.  Across  the 
weekly  discussion,  there  were  participants  who  raised  questions  and  asked  for  answers  and  others  who  actively  shared, 
exchanged,  and  constructed  new  ideas  and  concepts.  Both  types  of  interaction  in  the  discussion  were  conducive  to 
knowledge  construction. 

Many  researchers  (Rogoff,  1990;  Wertsch,  1985)  influenced  by  Vygotsky  have  studied  vertical  interaction 
(represented  by  adult-child  interaction).  A current  learning  model,  apprenticeship  (Collins,  Brown,  & Newman,  1989)  is 
also  concerned  with  interaction  that  is  vertical  in  nature;  however,  other  researchers  argue  that  the  construction  of 
knowledge  through  social  interaction  can  be  observed  much  more  often  in  horizontal  interaction  (peer  interaction)  (Hatano 
& Inagaki,  1991).  We  observed  the  same  phenomena  in  the  distance  learning  course.  The  horizontal  interaction  appeared 
much  more  often  than  the  vertical  interaction.  The  role  each  member  assumed  was  not  fixed  or  permanent,  but  could  be 
switched  and  interchanged  with  ease.  The  interaction  in  the  discussion  moved  naturally  and  smoothly  on  the  interaction 
continuum  rather  than  jumping  from  one  end  to  the  other.  That  is,  the  vertical-horizontal  interaction  can  be  seen  as  a 
continuum  rather  than  a dichotomy  (Hatano  & Inagaki,  1991). 

Third,  the  electronic  discussion  was  apparent  within  everyone's  zone  of  engagement,  therefore  everyone  was 
engaged  in  the  discussion,  rather  than  just  a few  students.  As  the  analysis  has  shown,  the  number  of  contributors  was  in 
appropriate  proportion  to  the  number  of  notes  each  week.  Most  students  contributed  once  or  twice  each  week.  Few 
students  contributed  more  than  twice.  Peers'  thoughts  and  notes  usually  created  cognitive  conflicts  or  perturbation  in  the 
students  and  urged  them  to  think  and  to  act.  Therefore,  VaxNotes  provided  a space  for  every  student  to  engage  in 
thinking,  acting,  expressing  ideas,  and  interacting  with  peers.  Moreover,  VaxNotes  served  as  an  excellent  place  for  them 
to  work  within  their  zone  of  proximal  development  (ZPD)  with  the  assistance  from  peers  and  instructors.  While  working 
within  their  own  ZPD  with  the  help  of  the  instructors  and  peers,  the  students  settled  many  cognitive  conflicts  and 
perturbations,  thereby  maximizing  cognitive  growth  and  development. 

Pattern  of  knowledge  construction  in  the  electronic  discussion 

The  electronic  discussion  in  VaxNote*  is  viewed  as  the  zone  of  engagement  and  development  (See  Figure  1)  in 
which  every  member  of  the  class  can  participate,  express  or  exchange  views,  and  actively  engage  in  constructing  a better 
understanding  of  issues  based  on  his  prior  knowledge  and  current  understanding. 
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Figure  1 

Pattern  of  knowledge  construction  in  the  electronic  discussion 


Note: 

The  core  ( inner  circle)  is  the  location  of  discussion  topics  . 

The  shaded  circle  (middle  circle)  contains  the  zones  of  engagement  and  development. 

The  white  circle  (outer  circle)  is  where  students  construct  new  knowledge, 
understanding,  insights,  and  perspectives. 

The  participant  category  circle  is  the  embedded  white  circle  with  the  abbreivation  of  PK' 
(Prior  knowledge). 
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New  perspective 


In  Figure  1,  the  construction  of  knowledge  starts  from  the  core,  that  is,  the  inner  circle- weekly  discussion 
topics  and  readings.  Surrounding  the  inner  circle  is  the  discussion  space  (the  shaded  middle  circle)-the  zone  of 
engagement  and  development  where  each  individual  was  involved.  After  numerous  trips  between  the  inner  circle  and  the 
middle  circle,  and  various  interactions  with  equal  and  superior  peers  and  instructors,  the  student  finally  arrives  at  the  outer 
circle  wherein  he/she  constructs  new  knowledge  and  understanding.  The  assistance  from  instructors  and  peers  and  the 
interaction  among  them  are  crucial  to  learning  and  knowledge  construction  (Gallimore  & Tharp,  1990).  The  constant 
trips  and  interactive  relationships  are  represented  by  bi-directional  arrows  used  between  the  inner  circle  and  the  middle 
circle  and  among  categories  of  participants  within  the  shaded  middle  circle.  In  the  electronic  discussion,  the  student  and 
the  instructor  may  assume  the  roles  of  contributor,  wanderer,  information-seeker,  and  mentor.  As  indicated,  the  rote  of 
each  individual  is  by  no  means  permanent  or  exclusive.  The  student  can  assume  several  different  roles  or  shift  roles  in 
one  electronic  note.  The  bi-directional  arrows  between  each  role  depicts  the  interchangeable  and  interactive  nature  of  a 
role. 

According  to  this  model,  a student  reads  weekly  assignments,  discusses  them  in  the  electronic  space,  and  reads 
other  students’  notes.  When  moving  back  and  forth  between  the  inner  circle  and  the  middle  circle-from  reading  and 
discussing,  the  student  brings  prior  knowledge  and  experience.  The  activities  of  reading  and  discussing  are  highly 
integrated  so  that  the  student  may  read  one  article  at  one  moment  and  in  the  next  minute  may  post  thoughts  in  the 
electronic  conference  or  read  notes  from  other  students.  Embedded  in  these  activities  are  a series  of  other  learning 
activities  such  as  sharing  information,  requesting  answers,  reflecting,  and  commenting.  Thus,  while  engaging  in  such 
activities,  the  student  also  is  assuming  the  responsibility  to  plan,  monitor,  motivate,  and  regulate  learning.  Exercising 
such  metacognitive  skills  has  received  increasing  attention  by  educators  and  researchers  (Paris  & Winograd,  1990). 

The  formation  of  a plan,  goal-setting,  and  other  activities  are  socially  and  cognitively  regulated  by  the 
interaction  with  peers  and  instructors.  From  Vygotsky’s  (1985)  point  of  view,  the  student  is  moving  between  the 
interpersonal  plane  (social  plane)  and  the  intrapersonal  plane  (psychological  plane)  in  the  electronic  discussion,  because 
activities  such  as  planning,  monitoring,  and  self-regulating  have  to  happen  on  both  planes.  In  other  words,  the 
individual's  cognitive  development  is  not  attributed  to  the  individual  alone  but  has  its  origins  in  social  groups  such  as  the 
large  and  small  discussion  groups  in  our  electronic  discussions.  The  student  has  to  adjust  and  re-define  learning  activities 
through  interactions  with  peers  and  instructors.  In  our  case,  the  notes  contributed  by  each  individual  urge  other  students 
to  re-examine  their  learning  goals  and  objectives.  These  notes  in  the  discussion  also  create  cognitive  conflicts  for  any 
future  reader.  In  other  words,  social  interaction  serves  as  a stimulus  for  individual  cognitive  growth. 

Though  knowledge  construction  and  acquisition  share  the  same  pattern  among  students  in  the  electronic 
discussion,  differences  do  exist  even  among  those  who  are  actively  involved  in  the  same  discussion.  The  acquisition  of 
new  knowledge  and  understanding  will  not  be  equal  across  the  board.  It  depends  on  the  amount  of  activities  in  which  the 
individual  is  engaged  in,  the  effort  invested,  and  on  his  or  her  prior  knowledge.  While  the  major  pattern  of  knowledge 
construction  in  the  electronic  conference  is  similar  (i.e.  starting  from  the  inner  circle  and  arriving  at  the  outer  circle  as 
illustrated  in  Figure  1),  the  sub-process  each  individual  engages  in  may  differ.  Two  possible  processes  could  be  identified 
in  the  knowledge  construction  in  the  electronic  discussion:  a)  individual  construction  of  knowledge  motivated,  influenced, 
and  facilitated  by  the  discussion  and  the  interaction  with  peers;  and  b)  assimilation  of  information  proposed  by  others 
with  some  individual  editing.  The  more  active  students  are,  the  more  likely  they  are  to  go  through  the  first  process  and 
the  less  active  students  engage  in  the  second  process. 

Electronic  Discussion  Organization 

As  indicated  earlier,  the  organization  of  the  electronic  discussion  consists  of  fr/tr.  major  components:  the  starter, 
the  wrapper,  the  introductory  questions  and  advice  from  the  instructor,  and  participant  comments.  A good  starter  usually 
pointed  to  a few  major  discussion  themes  for  a weekly  discussion.  However,  using  a vrapper  for  each  week  did  not 
reveal  many  educational  advantages.  In  fact,  most  students  simply  ignored  the  wrapper.  The  reason  may  be  that  the 
summary  of  one’s  learning  can  be  quite  individual.  When  a student  has  gone  through  a week's  discussion,  he/she  needs  to 
think  or  reflect  himself/herself  about  what  and  how  is  learned.  There  will  be  little  or  probably  no  commonality  in  the 
way  each  individual  summarizes  or  concludes  at  the  end  of  a weekly  discussion.  No  individual’s  wrap-up  can  be  so 
comprehensive  and  exhaustive  as  to  include  and  represent  the  ideas  of  the  entire  class.  From  this  perspective,  the  role  of 
wrapper  does  not  function  as  successfully  in  the  electronic  discussion  as  is  expected. 

The  instructor’s  introductory  questions  served  as  exemplary  guides  to  the  discussion.  However,  the  students 
usually  did  not  set  the  discussion  around  those  questions,  nor  did  the  instructors  endeavor  to  lead  the  discussion  around 
the  questions.  The  reason  for  this  could  be  that:  1)  students  do  not  see  any  close  relationships  between  the  readings  and 
the  questions  posted;  2)  the  students  feel  more  restricted  in  the  discussion  if  they  follow  the  questions;  3)  the  questions 
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are  too  broad  to  focus  on;  and  4)  the  instructors  do  not  have  the  adequate  mechanisms  to  lead  the  discussion  towards  the 
questions.  Questions  in  Week  7 and  Week  10  could  explain  one  point  here.  The  instructor  posed  such  introductory 
questions  for  Week  7:  "Why  is  information  access  so  important?  Can  we  teach  search  skills?"  For  the  discussion  in 
Week  10,  one  introductory  questions  was:  "Is  there  any  difference  between  tools,  tutors,  and  multimedia  word 
problems?".  These  questions  were  very  broad  and  hard  to  focus  on  given  no  further  guidance. 

Though  the  organization  of  the  electronic  conference  was  conducive  to  discussions  and  effective  in  facilitating 
learning,  much  could  be  improved.  First  of  all,  after  assigning  the  role  of  the  starter  and  the  wrapper,  mechanisms  could 
be  invented  to  constantly  monitor  the  implementation  of  roles  and  to  revise  them  if  necessary.  Secondly,  in  addition  to 
the  role  of  the  starter  and  the  wrapper  for  the  weekly  discussion,  volunteer  students  could  be  recruited  each  week  to  serve 
as  extra  facilitators  and  mentors  during  the  discussion.  The  instructors  could  have  paired  up  some  advanced  students  with 
new  students  and  let  them  serve  as  mentors  to  the  new  students.  When  in  a pair  or  a small  group,  advanced  students 
could  more  accurately  diagnose  the  new  students’  skill  and  understanding  levels,  thereby  providing  them  with  appropriate 
information,  sharing  experiences  with  them,  and  helping  them  better  understand  the  issues.  The  advanced  students  could 
serve  as  a bridge  enhancing  the  link  between  instructors  and  the  new  students.  While  these  students  are  still  novices 
compared  to  the  instructor,  they  are  perceived  as  experts  by  the  new  students.  They  could  play  dual  roles-help 
themselves  as  well  as  fellow  students  to  learn.  If  the  advanced  students  in  the  class  were  taking  dual  roles,  more  of 
students'  learning  could  be  guided  and  consequently  their  involvement  in  the  discussion  could  be  more  meaningful,  and 
their  learning  more  successful.  Lastly,  the  introductory  questions  for  the  discussion  could  be  more  issue-based  to  give 
students  more  latitude  in  developing  their  own  ideas  and  thoughts.  Meanwhile,  it  could  draw  them  to  central  issues  and 
thus  be  easy  for  the  instructors*  guiding  and  scaffolding. 

Roles  of  Students  and  Instructors 

Students'  role.  The  use  of  VaxNotes  and  its  discussion  has  reconfigured  the  role  of  the  student  and  the  instructor 
in  this  distance  learning  course.  Students  in  this  class  were  actively  involved  in  their  own  learning  through  such 
activities  as  planning,  self  monitoring,  and  self-regulating.  What  the  instructor  provides  here  was  reading  materials, 
insights,  and  an  electronic  conferencing  tool.  It  was  left  to  the  students  to  decide  how  to  take  advantage  of  the  tool  (i.e., 
VaxNotes),  the  electronic  resources,  and  assistance  from  the  instructors.  Students  decided  how  to  participate  in  the 
discussion,  how  to  learn,  and  what  to  learn.  Learning  was  centered  around  students  and  thus  they  had  more  control  over 
their  own  learning.  Some  students  enjoyed  using  VaxNotes  as  a tool  to  engage  in  the  discussion  because  they  believe 
the  action  of  writing  down  their  ideas  makes  them  think  and  reason  more  deeply  and  clearly.  Our  survey  indicated  that 
students  believe  that  through  this  electronic  conferencing  they  learn  more  about  the  subjects  and  about  the  peers; 
everyone  is  participating,  even  the  most  shy  students  whose  perspectives  and  voices  are  usually  absent  in  face  to  face 
discussion.  For  example,  one  student  said:  "Surely,  using  VaxNotes  or  news  groups  or  even  email  has  a big  advantage, 
that  is,  for  shy  and  foreign  students  whose  first  language  is  not  English,  this  is  a very  good  place  to  express  their 
opinion." 

Instructors’  role.  The  role  of  the  instructor  has  been  reconfigured  just  as  the  role  of  the  student  has  been.  The 
instructor  is  no  longer  a knowledge  dispenser  or  a lecturer  who  preaches  without  listening  to  students’  input, 
understanding,  and  interpretation  of  the  readings.  Instead,  in  this  course  the  instructor  was  the  mentor  and  the  facilitator 
of  students'  learning.  The  instructor  read  students’  notes,  interacted  with  them,  assessed  their  understanding,  and  thus 
provided  guidance  for  their  understanding  and  learning.  For  example,  in  Week  6,  one  instructor  explained  the  notion  of 
shared  space:  "Shared  space  is  the  place  where  you  can  go  to  communicate  and  you  and  others  can  come  back  to  it  and 
review  it.  With  technology,  shared  space  makes  thoughts  and  ideas  visible  through  email  or  a bulletin  board  system  like 
VaxNotes. ...  If  you  and  I were  working  on  a paper  and  you  left  a copy  of  it  on  the  network,  then  the  file  it  is  stored 
under  is  our  shared  space.  It  is  a place  we  can  go  to  when  conversing  and  reflecting  on  that  conversation." 

The  task  of  guiding  students’  learning  is  no  easy  job.  In  the  course,  the  instructors  invented  several  ways  to 
guide  students’  learning.  The  methods  of  guidance  used  in  the  electronic  discussion  are  quite  successful  on  the  whole. 
Most  students  expressed  positive  feelings  towards  the  electronic  discussion  and  their  learning  in  the  class.  One  student 
wrote: 


"I  admit  that  I dread  coming  to  the  lab  to  post  notes  and  reflections  about  the  articles  and  class  discussion. 
However,  like  Julie,  I recall  times  in  class  when  there  was  something  I wanted  to  discuss  but  for  one  reason  or 
another,  the  point  was  forgotten.  Using  electronic  classrooms  as  the  primary  means  for  discussion  would 
allow  us  to  explore  ideas  in  depth  and  several  ideas  at  the  same  time.  Students  who  have  a difficult  time 


O 

u 
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following  the  class  discussions  have  a chance  to  catch  up.  The  electronic  classroom  gives  them  a tool  for 

reflection  and  clarification  about  class  discussion.  Students  may  not  feel  as  apprehensive  about  discussing. 

That  is  a tool  that  benefited  all  of  us." 

Three  methods  of  guidance  have  been  invented  and  implemented  with  relative  successes  and  failures.  First  of  all, 
the  instructor  designed  the  roles  of  the  starter  and  wrapper  to  facilitate  the  electronic  discussion.  Also,  the  introductory 
discussion  questions  were  meant  to  guide  students'  discussion;  however,  most  students  disregarded  the  questions  because 
"they  are  too  hard,  long,  and  difficult  to  follow." 

A second  method  instructors  employed  to  guide  and  scaffold  students'  learning  was  to  build  upon  students'  notes. 
During  the  discussion,  the  instructor  added  comments  to  generate  more  conversation,  pushed  students  to  defend  their 
opinions,  and  answered  students'  information  seeking  questions.  Of  course,  due  to  the  larger  number  of  students  in  class, 
it  was  extremely  hard  for  the  instructor  to  meet  every  student's  learning  and  needs.  Some  students  felt  that  instructors 
could  add  more  of  their  experiences  and  thoughts  about  the  readings  and  could  answer  or  comment  more  on  the  questions 
raised  by  students.  Sometimes,  they  expected  less.  For  example,  one  student  wrote:  "I'd  like  for  the  comments  to  be 
less  all-encompassing.  Just  a nudge  here  or  there  would  be  fine.  What  the  instructor  says  tends  to  be  taken  as  the  final 
word  on  an  issue,  thus  it  can  stifle  conversation."  Another  student  expressed  the  opposite  view:  "...  I think  instructor 
should  pay  more  attention  to  lead  the  discussion  and  make  sure  everyone  catch  the  points." 

Finally,  the  instructors  utilized  the  method  of  role  play  during  the  electronic  conference.  Each  student  was  given 
a role  or  could  choose  one.  Some  students  assumed  the  role  play  very  well  and  motivated  other  peers  to  contribute  more. 
However,  the  role  play  method  was  not  well  accepted  by  some  other  students,  because  of  their  personal  learning  style. 
They  simply  felt  that  they  could  not  discuss  and  assume  one  of  the  roles  simultaneously. 

CONCLUSION. 

This  study  reveals  a promising  way  of  employing  a piece  of  ordinary  communication  technology  to  facilitate 
students'  learning  and  has  proven  to  be  quite  successful.  In  the  electronic  discussion,  students  selected  topics,  determined 
their  goals,  shared  experiences  and  understanding  with  peers.  Meanwhile,  they  tried  to  make  sense  of  the  issues  and  the 
concepts  through  reading  articles  and  working  within  their  own  ZPD  with  the  assistance  from  instructors  and  peers. 
VaxNotes  is  a "shared  space"  wherein  students  can  engage  in  various  collaborative  learning  activities.  This  is  a social 
and  intellectual  tool  for  facilitating  understanding  and  supporting  the  social  construction  of  knowledge. 

The  study  has  reported  some  pioneering  efforts  in  using  computer  mediated  communication  tools  in  instruction. 
More  empirical  research  should  be  conducted  to  verify  the  participant  and  note  coding  schemes  and  models  of  analysis 
used  here.  More  studies  need  to  be  conducted  to  establish  practical  guidelines  for  incorporating  computer  mediated  tools 
ir  ■)  classrooms,  because  before  one  can  begin  to  look  at  effective  implementation  of  a new  technology,  one  must 
understand  how  that  technology  can  be  used  both  mechanically  and  pedagogically.  The  field  is  promising  and  wide  open 
to  researchers  from  many  disciplines. 
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Figure  A 

Personal  Interests  Continuum 
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Lewin’s  Visualization  of  the  Life  Space  of  an  Individual 
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Figure  1.  Study  Activities 
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Figure  1.  TV  Viewing  System 
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Figure  4.  A Flowchart  of  the  Theory-Based  Procedure 
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ABSTRACT 


This  study  examined  how  the  use  of  generative  and  non-generai.  u visual 
manipulation  strategies  in  computer-based  instruction  and  level  of  prior  knowledge 
influenced  achievement  on  six  posttest  measures  representing  different  levels  of  learning 
objectives.  One-hundred-eighty-four  undergraduate  volunteers  were  administered  a pretest 
on  human  physiology  and  categorized  into  high,  medium  and  low  prior  knowledge  groups. 
Subjects  were  randomly  assigned  to  four  treatments  ( visual  summary  with  manipulation, 
learner  manipulation,  computer  manipulation,  and  a control)  based  on  their  prior 
knowledge  classification.  After  completion  of  the  instruction,  students  completed  a 
criterion-referenced  test. 

Analysis  of  variance  revealed  a significant  effect  for  prior  knowledge.  No  main 
effect  was  found  for  manipulation  and  a lack  of  interactive  effect  was  found  between  prior 
knowledge  and  manipulation.  Further  analyses  were  conducted  using  pairwise 
comparisons  to  detect  any  potential  treatment  effects.  Significant  differences  were  found 
for  the  control  treatment  over  learner  manipulation  of  visuals  in  examining  the  pairwise 
comparisons.  Similarly,  significant  differences  were  also  found  for  the  control  treatment 
over  computer  manipulation  of  visuals.  The  presentation  of  static  visual  and  textual 
information  prior  to  manipulation  of  visuals  indicates  a potential  variable  to  enhance 
learning  over  computer  manipulation  of  visuals,  while  the  control  also  seems  to  improve 
learning  over  computer-controlled  manipulation.  Exploratory  analysis  revealed  results 
similar  to  the  overall  analyses.  In  addition,  trends  among  mean  scores  supported  the 
overall  results  and  showed  the  control  as  exhibiting  the  highest  mean  scores  on  most 
measures  for  both  high  and  low  prior  knowledge  learners. 

This  study  supports  previous  research  that  prior  knowledge  is  a primary  factor  in 
learning.  Those  with  high  prior  knowledge  out-performed  students  with  low  prior 
knowledge  on  all  measures.  This  research  also  suggests  that  the  manipulation  of  visuals 
onto  a visual  framework  may  make  the  recall  of  factual  and  conceptual  information  more 
difficult.  Presenting  static  visual  and  textual  information  prior  to  manipulation  of  visuals 
may  potentially  enhance  learning  on  tasks  requiring  the  learner  to  reconstruct  a visual 
representation  of  concepts.  However  it  is  speculated  that  additional  training  with 
manipulation  strategies  should  be  presented  to  students.  Further  research  with  manipulation 
of  visuals  may  help  to  determine  the  most  appropriate  methods  for  assisting  learners  in 
acquiring  information  in  computer-based  instruction. 
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INTRODUCTION 


The  necessity  of  the  student's  active  role  in  the  learning  process  has  been  well 
established  by  the  cognitive  movement  in  education.  Designing  instruction  in  a way  that 
facilitates  learners'  meaningful  interaction  with  and  processing  of  information  is  a 
continuing  challenge  for  instructional  designers.  Wittrock's(  1974a;  1974b;  1990) 
generative  learning  theory  presents  a model  for  directing  learner's  cognitive  processes  to 
establish  links  to  memory  through  the  active  construction  of  relationships  and  meaning. 
Activities  such  as  the  manipulation  and  organization  of  instructional  elements  by  the  learner 
can  initiate  the  encoding  process  and  have  been  shown  to  increase  achievement  in  several 
contexts  (Wittrock  & Carter,  1975;  Linden  & Wittrock,  1981;  Dee-Lucas  & DiVesta, 

1980).  However,  the  generative  manipulation  and  organization  of  visual  elements  in 
computer-based  instruction  is  a fairly  unexplored  area  of  empirical  investigation. 

Through  generative  interaction,  the  learner  constructs  relationships  among 
instructional  elements  or  between  the  elements  and  his  or  her  prior  knowledge. 

Grabowski  (1995,  in  press)  distinguishes  these  two  types  of  generative  processing  as 
organizational  (constructing  relationships  among  the  parts)  and  integrational  (constructing 
relationships  among  the  instruction  and  prior  knowledge).  Generative  activities  involving 
visual  elements  have  generally  included  student  construction  of  pictures,  drawings  and 
mental  images  and  have  facilitated  factual  retention  of  definitions  and  enhanced  reading 
comprehension  (Linden  & Wittrock,  1981;  Bull  and  Wittrock,  1973).  By  creating  visual 
representations,  learners  generate  relationships  between  parts  of  the  text  to  one  another  or 
between  the  text  and  their  prior  knowledge. 

In  addition  to  creating  visual  illustrations  or  drawings,  the  manipulation  and 
organization  of  existing  visual  elements  may  also  induce  generative  processing. 
Manipulation  of  concrete  visual  elements  or  manipulative  materials  have  been  widely  used 
as  instructional  tools  in  the  mathematics  and  science  domains  (Gabel  & Sherwood,  1980; 


Goodstein  & Howe,  1978).  Manipulative  objects  are  used  to  encourage  increased 
understanding  through  the  transformation  of  concrete  concepts  to  semi-concrete 
representations  and  finally,  to  the  internalization  of  concepts  through  abstract  thought 
(Berlin  & White,  1986) . However,  certain  types  of  manipulative  objects  may  be  better  in 
facilitating  achievement  than  others  (Sayeki,  Ueno  & Nagasaka,  1991).  Manipulation  of 
visual  objects  on  the  computer  screen  has  traditionally  been  defined  as  animated  or  dynamic 
visuals-  those  whicli  move  under  the  control  of  a computer  or  video  delivery  system.  Park 
and  Hopkins  (1992)  specify  dynamic  visual  displays  as  any  type  of  graphical  movement 
during  instruction.  Their  review  of  studies  in  this  area  indicate  mixed  results  regarding  the 
instructional  effectiveness  of  dynamic  visual  displays.  Milheim  (1993)  elaborates  on  the 
disadvantages  of  animated  visual  displays  by  highlighting  the  minimal  interaction  with  the 
learner  this  type  of  presentation  affords. 

In  contrast  to  computer-controlled  dynamic  visual  displays,  current  development 
tools  allow  for  the  learner  to  interact  with  graphics.  The  computer  interface  described  as 
the  "direct  manipulation  interface,"  allows  the  learner  to  be  directly  engaged  with  visual 
objects  on  the  screen,  rather  than  indirect  interaction  through  programming  (Hutchins, 
Hollan  & Norman,  1986).  The  movement  and  placement  of  visuals  on  the  computer  screen 
into  a visual  organization  may  induce  generative  processing.  However,  control  of  the 
manipulation,  either  by  the  learner  or  by  the  computer,  may  further  account  for  the 
instructional  effectiveness  of  this  strategy. 

Alesandrini  (1984)  promotes  manipulation  of  visual  elements  as  a potential 
instructional  strategy,  indicating  that  what  seems  to  be  important  is  the  act  of  manipulation 
itself.  Scheiderman  (1992)  contends  that  physical,  spatial  or  visual  representations  of 
information  appear  to  be  easier  to  retain  and  manipulate  than  textual  or  numeric 
representations.  Hennessey  and  O'Shea  (1993)  associate  manipulation  of  computer 
representations  with  concrete  objects  in  stating  that  while  the  instructional  advantages  to 
manipulating  concrete  objects  has  been  well  established,  similar  advantages  of  manipulating 


on-screen  representations  have  not  been  well  researched.  Through  the  active  manipulation 
of  computer  objects  into  a visual  organization,  the  learner  may  consciously  construct 
associations  between  the  visual  elements.  This  type  of  activity  has  the  potential  to  provoke 
generative  processing. 


PROBLEM  STATEMENT 

Manipulation  of  visuals  in  computer-based  instruction  may  assist  the  learner  in 
creating  necessary  and  appropriate  associations  among  presented  instructional  stimuli. 
However,  control  of  the  act  of  manipulation-by  the  computer  or  the  learner—  may  impact 
the  level  of  cognitive  processing  which  occurs.  Additionally,  the  learner’s  level  of  prior 
knowledge  may  impact  what  is  learned.  Finally,  the  learner’s  prior  knowledge  level  may 
also  influence  his  or  her  ability  to  capitalize  on  manipulation  as  a generative  learning 
strategy. 

Accordingly,  the  following  questions  were  addressed  in  this  study.  Will  the 
manipulation  of  visuals  allow  the  learner  to  construct  meaningful  relationships  among  the 
presented  instructional  components?  Does  the  active,  overt,  physical  manipulation  of  on- 
screen visual  elements  by  the  learner  induce  generative  processing?  Does  level  of  prior 
knowledge  impact  learning?  Finally,  is  the  manipulation  and  reorganization  of  visuals,  as  a 
generative  learning  strategy,  appropriate  for  learners  with  certain  levels  of  prior  knowledge 
(high,  medium  or  low)  in  recalling  specific  facts  and  concepts?  This  study  examined  the 
relationship  between  three  different  strategies  for  manipulating  visuals  -two  generative  and 
one  non-generative  type  of  manipulation,  learners'  prior  knowledge  and  achievement  on  six 
measures  demonstrating  different  levels  of  learning  objectives. 


METHOD 


Subjects  and  Lesson  Content 

This  study  involved  1 84  subjects  recruited  from  three  undergraduate  statistics 
courses  at  a large,  eastern  state  university.  The  participants  were  administered  a paper  and 
pencil  36-item  pretest  and  were  categorized  into  high,  medium  and  low  prior  knowledge 
groups.  Stratified  randomization  was  then  used  to  randomly  assign  groups  to  conditions. 

The  instructional  materials  used  in  this  study  were  adapted  from  paper-based  text  materials 
developed  by  Dywer  and  Lamberski  (1977)  on  the  physiology  and  function  of  the  human 
heart.  The  lessons  included  factual,  conceptual  and  problem  solving  information  and  was 
re-created  in  computer-based  instruction  format  using  Authorware  Professional. 

Treatment  Materials 

Four  computer-based  treatments  were  developed  for  this  study  which  contained 
identical  instructional  content  while  varying  the  type  and  presence  of  manipulation. 

Visual  Summary  with  Manipulation.  Subjects  in  the  visual  summary  with 
manipulation  (VS-M)  group  viewed  graphics  with  specific  parts  of  the  heart  highlighted 
and  corresponding  textual  information.  The  subjects  were  required  to  click  continue  to 
proceed  through  the  majority  of  computer  screens  which  contained  static  visual  and  text 
information.  At  five  points  in  the  instruction,  the  learner  was  presented  with  smaller 
graphical  representations  of  four  parts  of  the  heart  reviewed  in  that  section  on  the  left  side 
of  the  screen  and  a target  box  on  the  right  side  (See  Figure  1).  Using  these  parts,  the 
learner  then  constructed  their  own  visual  summary  of  the  information  by  clicking,  dragging 
and  organizing  the  parts  into  the  box  based  on  what  they  had  learned.  Correct  placement  of 
these  parts  was  not  judged  and  learner  needed  only  to  place  all  parts  in  the  box  in  order  to 
continue  (See  Figure  2).  The  organization  of  visuals  depicted  a generative  representation  or 
visual  summary  of  the  lear  er's  understanding  of  the  previous  information. 
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Learner-Manipulation.  Subjects  in  the  learner-manipulation  (L-M)  group  were 
required  to  click  and  drag  a visual  graphic  of  a part  of  the  heart  from  the  left  side  of  the 
screen  onto  a visual  frame  of  the  whole  heart  containing  an  outline  of  the  primary  parts  (See 
Figure  3).  No  text  was  i rented  prior  to  the  movement,  forcing  the  learner  to  first 
discover  the  correct  position  on  the  heart.  If  the  part  was  not  placed  correctly,  it 
automatically  returned  to  its  original  position  in  the  left  hand  comer  of  the  screen,  providing 
animated  negative  visual  feedback.  The  subject  was  required  to  place  the  visual  again  and 
was  permitted  three  attempts  after  which  the  part  would  animate  to  its  correct  position. 

After  successful  placement  of  the  part,  textual  information  describing  the  part,  its  location, 
function  and/or  relationship  to  other  parts  of  the  heart  appeared  next  to  the  visual  (Figure 
4).  This  activity  represented  a generative  approach  where  subjects  would  mentally 
construct  associations  among  the  presented  visuals  through  manipulation. 

Computer-manipulation.  In  the  computer-manipulation  (C-M)  treatment,  the 
learner  was  presented  with  a part  of  the  heart  on  the  left  side  of  the  screen  and  an  identical 
visual  frame  of  the  whole  heart  as  presented  in  the  learner-manipulated  treatment  as  shown 
in  Figure  3.  However,  in  the  C-M  treatment,  the  learner  did  not  click  and  drag  the  part,  but 
instead  clicked  continue  which  caused  the  part  to  animate  to  the  correct  location.  At  that 
point,  the  textual  information  describing  the  part,  its  location,  function  and/or  relationship 
to  other  parts  of  the  heart  appeared.  This  activity  represented  a non-generative  form  of 
manipulation  of  visuals. 

Control.  The  control  group  presented  the  lesson  content  via  text  and  graphics  in  a 
page-turning  format  with  no  learner  manipulation  of  visuals.  Subjects  viewed  graphics 
with  specific  parts  of  the  heart  highlighted  and  their  corresponding  textual  information 
(Figure  5).  The  only  action  required  by  the  learner  was  to  click  continue  to  proceed. 
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Dependent  Measures  and  Reliability 


The  posttests  used  in  this  study  consisted  of  paper  and  pencil  tests  encompassing  a 
total  of  100  questions.  Generally,  the  five  tests  were  designed  to  measure  the  ability  of  the 
student  to  identify  individual  parts  of  the  heart  using  visual  cues,  retain  facts,  terms  and 
definitions  related  to  the  heart  as  well  as  utilize  information  for  problem  solving.  Each  test 
consisted  of  twenty  multiple-choice  questions. 

Specifically,  the  identification  test  evaluated  retention  of  factual  information  and 
required  students  to  identify  the  names  of  parts  on  drawings  of  the  heart.  The  terminology 
test  was  designed  to  measure  knowledge  of  facts,  terms  and  definitions.  The 
comprehension  test  measured  the  ability  of  the  student  to  use  the  information  to  explain 
another  phenomenon  occurring  simultaneously.  The  problem  solving  test  assessed  the 
student’s  ability  to  apply  their  understanding  of  the  information  in  reasoning  about 
alternative  situations.  The  identification,  terminology,  comprehension  and  drawing  tests 
required  the  learner  to  view  visual  or  draw  representations  of  the  heart  and  select  the  correct 
location  or  position  of  parts  while  the  problem  solving  test  was  presented  through 
comprehension  of  textual  information.  The  total  written  test  highlighted  the  students 
overall  performance  in  combining  all  measures  except  the  drawing  test  reflecting  a 
composite  score. 

Internal  consistency  measures  were  computed  for  each  of  the  tests  using  Cronbach 
alpha  reliability  coefficient  which  is  equivalent  to  the  KR-20  (Kuder-Richardson-20).  The 
following  Cronbach  alpha  coefficients  were  found:  .79  Identification,  .75  terminology, 

.74  comprehension,  .70  problem  solving,  .79  total  test,  and  .83  Drawing. 

i 

Procedures 

The  study  was  introduced  to  students  during  regular  sessions  of  their  respective 
statistics  course.  To  divide  the  groups  into  high,  medium  and  low  prior  knowledge  levels, 
a technique  which  has  been  used  in  at  least  one  previous  study  involving  the  same 


instructional  materials  was  used  (Akanbi,  1986).  Subjects  were  classified  as  high,  medium 
or  low  prior  knowledge  based  on  whether  their  pretest  score  was  more  than  one  half  a 
standard  deviation  above  the  mean,  or  between  one  half  standard  deviation  above  and 
below  the  mean  or  more  than  one  half  a standard  deviation  below  the  mean.  The  pretest 
results  were  used  as  an  scheme  to  randomize  subjects  from  all  three  classes  into  high, 
medium  and  low  prior  knowledge  groups. 

Treatments  were  conducted  in  one  of  the  University  computer  labs.  Upon 
completion  of  the  instruction,  posttests  were  immediately  administered  in  a nearby 
classroom  in  the  following  succession:  drawing,  identification,  terminology, 
comprehension  and  problem-solving  tests.  The  treatments  were  presented  via  computer 
while  the  pretest  and  final  testing  were  delivered  in  traditional,  paper  and  pencil  format. 

Research  Design  and  Data  Analysis 

This  study  employed  a post-test  only  control  group  design  and  stratified  random 
sampling.  Stratified  sampling  was  used  in  order  to  examine  differences  that  might  exist 
between  subgroups  of  the  subject  population.  The  basis  for  stratificat,  n in  this  study  was 
level  of  prior  knowledge  (high,  medium  and  low)  which  was  assessed  by  the  pretest  on 
human  physiology. 

A two-factor  analysis  of  variance  (ANOVA)  was  used  to  test  for  interaction  effects 
between  the  factors  (prior  knowledge  and  type  of  manipulation)  and  for  significant  main 
effects  among  the  four  groups  for  each  criterion  measure:  drawing,  terminology, 
identification,  comprehension,  problem  solving  and  total  written  test  which  included  a 
composite  score  of  all  tests  except  drawing.  In  all  cases,  .05  alpha  level  was  selected  to 
determine  significance. 
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RESULTS 


Descriptive  results  for  achievement  showing  the  mean  scores  and  standard 
deviations  for  the  four  treatments  for  each  immediate  posttest  are  shown  in  Table  1. 
Differences  in  performance  levels  of  the  four  treatment  groups  showed  little  variance  in 
mean  scores  across  all  prior  knowledge  levels  on  each  of  the  posttests.  The  control  group 
showed  the  strongest  performance  on  all  criteria  except  problem-solving  and  drawing 
where  visual  summary  with  manipulation  (VS-M)  recorded  the  highest  mean  scores, 
although  not  a dramatic  increase.  Students  in  the  computer  manipulation  group  (CM) 
demonstrated  the  lowest  mean  scores  on  all  measures  except  terminology  and  problem 
solving. 

Table  2 shows  mean  scores  and  standard  deviations  for  three  levels  of  prior 
knowledge  for  each  criterion  measure  across  all  treatment  groups.  The  high  prior 
knowledge  group  scored  consistently  higher  on  each  posttest  measure  than  the  medium 
prior  knowledge  group,  followed  by  low  prior  knowledge. 

An  analysis  of  variance  was  run  on  each  of  the  dependent  measures  as  shown  in 
Table  3.  ANOVA  revealed  a significant  main  effect  on  all  measures  for  prior  knowledge 
while  the  main  effects  for  the  factor  of  manipulation  were  not  significant  across  all 
measures.  Results  also  revealed  no  interactive  effect  between  the  factors  of  prior 
knowledge  and  the  manipulation  conditions  on  all  measures. 

Students  with  high  prior  knowledge  scored  significantly  better  than  those  with  low 
prior  knowledge  on  all  measures.  The  performance  of  those  students  with  medium  level  of 
prior  knowledge  differed  significantly  from  those  with  low  on  the  terminology, 
comprehension,  problem  solving  and  total  written  tests. 

On  the  problem  solving  measure,  all  three  levels  of  prior  knowledge  differed 
significantly  from  one  another  with  the  high  level  group  (M=12.8)  performing  better  than 
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the  medium  group  (M=  10.64)  and  better  than  the  low  group  (M=9.31).  Similarly,  for  the 
total  written  test  which  subsumes  all  tests  except  the  drawing  measure,  the  results  indicated 
significant  differences  between  the  high,  medium  and  low  prior  knowledge  groups  with 
means  of  17.14,  15.92,  and  14.65  respectively. 

Although  the  F-tests  for  each  measure  examining  treatment  effects  for  manipulation 
were  not  significant  overall,  additional  analyses  examining  pairwise  comparisons  were 
conducted  in  order  to  further  investigate  the  data.  In  the  subsequent  discussion,  these 
results  merely  suggest  an  effect  for  the  treatments  which  may  have  been  masked  by  the 
overall  effect  of  both  treatment  and  prior  knowledge. 

While  the  overall  ANOVA  was  not  significant  on  the  identification  measure, 
differences  were  noted  among  the  pairwise  comparisons  of  learner  manipulation  and 
control  group.  There  were  notable  differences  indicated  between  the  control  group 
(M=  16.46)  which  scored  higher  than  the  learner  manipulation  (M=15.26)  and  computer 
manipulation  group  (M=  14.95).  Further  analysis  on  the  terminology  test  indicated  that  the 
control  group  (M=13.84)  outperformed  the  learner  manipulation  group  (M=12.34).  On  the 
comprehension  test,  the  control  group  (M=  14.28)  outperformed  both  the  visual  summary 
with  manipulation  (M=12.83)  and  computer  manipulation  groups  (M=12.66).  In  addition, 
for  the  total  written  measure,  the  control  group's  (M=55.80)  performance  exceeded  the 
learner  manipulation  (M=51.34)  and  computer  manipulation  groups  (M=50.78). 

In  contrast  to  the  other  results,  on  the  drawing  test  the  visual  summary  with 
manipulation  group  (M=  16.731  outperformed  the  computer  manipulation  group 
(M=14.85).  However,  similar  to  the  other  analyses,  the  control  group  scored  higher  than 
the  computer  manipulation  group  (M=  14.85). 
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CONCLUSIONS 


This  study  investigated  the  effects  of  visual  manipulation  strategies  as  well  as  the 
effects  of  three  levels  of  prior  knowledge  on  learner  achievement  on  six  types  of  learning 
objectives.  The  manipulation  strategies  represented  two  forms  of  generative  manipulation 
and  a computer-controlled  or  non-generative  manipulation  strategy  in  computer-based 
instruction.  The  results  generally  indicated  that  students  who  interacted  with  the  control 
treatment  performed  better  than  those  who  interacted  with  the  manipulation  strategies  for 
most  measures  as  evidenced  by  higher  mean  scores.  However,  the  differences  between  the 
groups  were  not  statistically  significant. 

As  expected,  learners'  level  of  prior  knowledge  was  found  to  significantly  influence 
their  performance  on  all  posttests.  Students  who  were  categorized  as  having  high  prior 
knowledge  significantly  outperformed  students  who  were  categorized  as  having  low  prior 
knowledge  on  all  posttest  measures.  Additionally,  no  significant  interaction  effect  was 
found  between  the  factors  of  prior  knowledge  and  manipulation. 

The  overall  findings  of  this  study  suggest  that  the  manipulation  of  visuals  in 
computer-based  instruction  constitutes  a novel,  unconventional,  demanding  and  possibly  a 
distracting  generative  strategy  for  students.  While  generative  learning  theory  provides  a 
strong  theoretical  foundation  for  this  strategy,  the  results  clearly  reflect  the  need  for 
students'  increased  familiarity  and  practice  with  this  type  of  instructional  approach.  In  this 
regard.  Hooper,  Sales  & Rysavy  (1994)  have  suggested  that  students  need  to  practice 
using  generative  learning  strategies  before  they  can  use  them  effectively.  Mayer 
(1984)  elaborates  on  this  position  in  regard  to  organizational  tasks  by  maintaining  that  the 
building  of  internal  connections  between  instructional  stimuli  and  learners'  knowledge  may 
be  enhanced  by  explicit  training  in  the  organization  of  items  for  meaningful  learning  tasks. 

The  need  to  train  students  to  capitalize  on  generative  strategies  is  even  more 
pronounced  when  they  interact  with  visual  elements  in  a novel  fashion  as  was  required  in 
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this  study.  Traditional  methods  of  instruction  typically  involve  the  simultaneous 
presentation  of  static  visual  and  textual  information.  As  a result,  learners  may  be 
conditioned  to  this  type  of  information  presentation  interacting  with  static  displays  of 
images.  With  increased  exposure,  students  perhaps  become  more  visually  literate  in 
acquiring  information  and  constructing  meaning  from  this  static  visual  presentation  of 
information.  Consequently,  learners  may  tend  to  rely  upon  these  established  and  well- 
honed  strategies  for  learning,  rather  than  attempting  to  cognitively  interact  with  unfamiliar 
methods  of  manipulating  visual  information. 

The  students  in  this  study  physically  interacted  with  the  visual  elements,  but  may 
not  have  been  cognitively  engaged  in  constructing  relationships  among  the  visuals  and 
textual  information  during  the  manipulation.  Generative  connections  may  have  been 
formed  among  the  visual  parts  and  their  spatial  relationships,  but  not  constructed  between 
the  visuals  and  the  textual  information.  In  all  three  manipulation  treatments,  the  learner  was 
presented  with  a screen  of  visual  and  textual  information  that  was  equivalent  to  the  control 
condition,  after  the  visual  elements  were  manipulated.  Instead  of  directing  attention  toward 
forming  generative  relationships  among  the  instructional  elements,  the  manipulation  of  the 
visuals  may  have  delayed  learners  from  focusing  on  or  processing  the  static  visual  and 
textual  information  until  after  the  visuals  were  placed  in  position  at  which  time  they  reverted 
back  to  familiar  strategies  for  learning  the  content. 

In  addition,  the  manipulation  and  reorganization  of  visual  information  may  make  the 
recall  of  specific  facts  and  concepts  more  difficult  for  some  learners.  This  point  raises  a 
concern  for  the  necessity  of  learners'  metacognitive  awareness  of  generative  strategies  and 
their  instructional  purpose.  Although  a practice  screen  was  presented  to  the  student  prior  to 
the  instruction  in  this  study,  the  purpose  of  the  screen  was  primarily  for  students  to  practice 
the  mechanics  of  manipulating  visuals  in  order  to  move  through  the  lesson.  Perhaps  the 
subjects  in  this  study  needed  a clearer  understanding  of  the  instructional  purpose  of  the 
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manipulation  strategies,  such  as  suggestions  on  how  to  process  the  information  while 
manipulating  the  visual  information. 

Important  conclusions  from  this  study  also  involve  the  amount  and  order  of 
presenting  instructional  information  prior  to  the  learner's  engagement  in  a generative  task 
involving  the  manipulation  of  visuals.  Mayer  and  Anderson  (1992)  recommend  presenting 
animated  visuals  and  textual  information  contiguously  in  time  and  space.  This  study 
supports  the  notion  that  an  adequate  amount  of  visual  and  textual  information  may  need  to 
be  presented  in  a traditional  fashion  prior  to  generative  interaction  with  visual  elements  for 
particular  learning  objectives  rather  than  presenting  the  visual  information  and  then  text. 

The  learners'  level  of  prior  knowledge  greatly  influenced  performance  on  the 
posttests.  Significant  differences  in  performance  were  detected  between  distinct  groups  of 
students  classified  into  high,  medium  and  low  prior  knowledge  levels  on  most  measures. 
The  influence  of  this  factor  was  not  unexpected  based  on  previous  studies  of  this  effect 
(Chase  & Simon,  1973;  Chi,  1978;  Chiesi,  Spilich,  & Voss,  1979;  Spilich,  Vesonder, 
Chieii,  & Voss,  1979;  Voss,  1977).  Their  research  has  shown  that  the  knowledge  an 
individual  brings  to  the  learning  situation  is  a primary  factor  influencing  learning.  This 
study  clearly  supports  this  research  in  that  the  learner's  prior  knowledge  provided  a basis 
for  the  assimilation  and  accommodation  of  new  information  into  the  learner's  schema. 

Additional  results  indicated  that  there  were  no  significant  interactions  between  prior 
knowledge  and  treatment  on  all  measures.  It  was  expected  that  those  students  with  low 
prior  knowledge  would  benefit  from  the  generative  manipulation  strategy,  while  those  with 
high  prior  knowledge  would  not  require  additional  strategies  for  learning.  Although  at  least 
one  study  has  shown  an  interaction  between  prior  knowledge  and  generative  strategies 
(Peper  & Mayer,  1986a),  it  was  not  apparent  in  this  experiment.  These  results  may  be  due 
to  the  fact  that  students  may  not  have  processed  information  based  on  the  manipulative 
task,  thereby  rendering  the  strategy  inefficient  to  impact  the  retention  of  information  at  any 
level  of  prior  knowledge. 


In  examination  of  the  mean. scores,  students  with  high  and  low  prior  knowledge 
benefited  from  the  control  condition.  These  results  support  other  findings  that  perhaps 
generative  strategies  are  not  necessary  for  those  with  high  prior  knowledge  and  students 
with  low  prior  knowledge  may  benefit  from  additional  practice  with  these  strategies. 

Trends  among  the  mean  scores  also  suggest  manipulating  visuals  into  a summary  after 
viewing  static  visual  and  textual  information  may  potentially  be  as  effective  as  traditional 
fomis  of  information  presentation  for  learners  with  high  and  medium  prior  knowledge  on 
higher  level  learning  objectives. 

Although  many  of  the  above  findings  are  only  speculative,  further  research  is 
needed  to  support  the  use  of  manipulation  as  a generative  learning  strategy.  This  study  has 
contributed  information  for  the  use  of  a novel  strategy  in  computer-based  instruction.  The 
results  indicate  the  manipulation  of  visuals  holds  promise  for  increasing  the  level  of 
processing  induced  by  the  learner  when  interacting  with  visual  information.  However, 
only  with  future  research  can  the  appropriate  and  effective  use  of  manipulation  strategies  be 
revealed.  Future  results  should  determine  if  metacognitive  training  is  required,  if  the  use  of 
visual  manipulation  tasks  more  suited  to  the  cognitive  outcome  can  facilitate  learning  or  if 
different  types  of  manipulation  strategies  can  facilitate  achievement. 
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Table  1 


Descriptive  Data  for  Treatments  on  Each  Criterion  Measure  . 


VS-M 

n=42 

LM 

n=50 

CM 

n=47 

Control 

n=45 

Identification 

M 

15.97 

15.26 

14.91 

16.46 

SD 

(2.83) 

(3.21) 

(3.18) 

(2.56) 

Terminology 

M 

12.92 

12.34 

12.76 

13.84 

SD 

(3.59) 

(3.82) 

(2.85) 

(2.76) 

Comprehension 

M 

12.83 

13.08 

12.66 

14.28 

SD 

(3.67) 

(3.75) 

(3.74) 

(2.73) 

Problem-Solving 

M 

11.52 

10.66 

10.87 

11.20 

SD 

(3.69) 

(3.87) 

(3.48) 

(3.23) 

Total  Written 

M 

53.26 

51.34 

50.78 

55.80 

SD 

(11.87) 

(12.86) 

(11.30) 

(9.14) 

Drawing 

M 

16.73 

15.86 

14.85 

16.62 

SD 

(4.44) 

(4.23) 

(4.46) 

(3.22) 

SI 


/ t 
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Table  2 


Descriptive  Data  for  Prior  Knowledge  Leve1  on  Each  Criterion  Measure 


High 

Medium 

Low 

n=70 

n=56 

n=58 

Identification 

M 

16.44 

15.48 

14.79 

SD 

(2.92)  • 

(2.82) 

(3.14) 

Terminology 

M 

13.98 

13.14 

11.51 

SD 

(3.41) 

(2.95) 

(3.06) 

Comprehension 

M 

14.32 

13.25 

11.82 

SD 

(3.62) 

(3.37) 

(3.13) 

Problem-Solving 

M 

12.80 

10.64 

9.31 

SD 

(3.37) 

(3.38) 

(3.01) 

Total  Written 

M 

57.55 

52.16 

47.44 

SD 

(11.71) 

(10.32) 

(9.87) 

Drawing 

M 

17.14 

15.92 

14.65 

SD 

(4.00) 

(3.91) 

(4.23) 
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Table  3 


ANOVA  Tables  for  All  Measures 
ANOVA  Table  for  Identification 


df 

Sums  of  Squares 

Mean  Sq. 

F-Value 

P-Value 

Manipulation 

3 

66.577 

22.192 

2.549 

.0575 

Prior  Knowledge 

2 

81.317 

40.659 

4.669 

.0106* 

Manip.  * Prior  Know. 

6 

28.11 

4.685 

.538 

.7788 

Residual 

172 

1497.684 

8.707 

ANOVA  Table  for  Terminology 

df 

Sums  of  Squares 

Mean  Sq. 

F-Value 

P- Value 

Manipulation 

3 

55.361 

18.454 

1.853 

.1393 

Prior  Knowledge 

2 

194.974 

97.487 

9.791 

<.0001** 

Manip.  * Prior  Know. 

6 

53.462 

8.910 

.895 

.4999 

Residual 

LZ2_ 

1712.495 

9.956 

ANOVA  Table  for  Comprehension 
df 

Sums  of  Squares 

Mean  Sq. 

F-Value  P- Value 

Manipulation 

3 

71.649 

23.883 

2.080  .1046 

Prior  Knowledge 

2 

174.084 

87.042 

7.581  .0007** 

Manip.  * Prior  Know. 

6 

54.418 

9.070 

.790  .5789 

Residual 

172 

1974.710 

11.481 

819 
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Table  3 continued 


ANOVA  Tables  for  All  Measures 

x 

x 

ANOVA  Table  for  Problem  Solving 


df 

Sums  of  Squares 

Mean  Sq. 

F-Valu4  P- Value 

Manipulation 

3 

30.750 

10.250 

.958 

.4139 

Prior  Knowledge 

2 

415.415 

207.708 

19.416 

<0001*** 

Manip.  * Prior  Know. 

6 

66.080 

11.013 

1.030 

.4078 

Residual 

172 

1840.011 

10.698 

ANOVA  Table  for  Total  Written 

df 

Sums  of  Squares 

Mean  Sq. 

F-Value 

P- Value 

Manipulation 

3 

723.664 

241.221 

2.094 

.1028 

Prior  Knowledge 

2 

3173.744 

1586.872 

13.776 

<.0001*** 

Manip.  * Prior  Know. 

6 

419.145 

69.858 

.606 

.7249 

Residual 

172 

19812.306 

115.188 

ANOVA  Table  for  Drawing 

df 

Sums  of  Squares 

Mean  Sa. 

F- Value 

P- Value 

Manipulation 

3 

91.256 

30.419 

1.860 

.1382 

Prior  Knowledge 

2 

197.900 

98.950 

6.051 

.0029* 

Manip.  * Prior  Know. 

6 

60.485 

10.081 

.616 

.7169 

Residual 

172 

2812,589 

16.352 

*p<.05 

**p<.01 

***p<.0001 
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Figure  1.  Visual  Summary  with  Manipulation  Treatment 
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Figure  4.  Completed  Learner  Manipulation  928 
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Figure  5.  Control  Treatment 
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Utilizing  the  Concerns-Based  Adoption  Model  to 
Facilitate  Systemic  Change 

We  have  become  a technologically  based  society.  Every  facet  of  our  lives  requires  us 
to  interact  with  technology  in  some  form  or  manner.  The  economic  structure  of  the  United 
States  has  shifted  from  an  industrial  economy  to  a technology  based  economy.  The 
tremendous  growth  and  evolution  of  technology  has  created  challenges  for  education.  In  our 
society,  technology  has  moved  from  being  a “nice  to  have”  option  to  a necessity. 

Education  is  being  forced  to  keep  up  with  the  increasing  demands  placed  on  the 
workforce.  There  has  been  a call  for  systemic  restructuring  of  our  educational  system,  so  that 
it  can  better  meet  these  demands.  These  ever  increasing  demands  place  an  added  burden 
on  schools,  educators  and  instructional  designers.  Today's  job  market  places  a premium  on 
people  who  are  motivated,  self-directed  learners.  Educators  and  instructional  designers  need 
to  work  together  to  equip  students  with  the  intellectual  tools,  self-beliefs,  and  self-regulatory 
processes  which  will  enable  them  to  meet  the  needs  of  employers  by  becoming  self-directed 
learners  (Bandura,  1993). 

In  restructuring  education,  technology  will  become  a vital  tool  for  helping  students 
build  self-regulatory  skills,  enhancing  the  productivity  of  the  learning  environment  and  the 
quality  of  the  experience.  Technology  has  been  defined  as  the  use  of  scientific  knowledge  to 
solve  educational  problems  (Anglin,  1995).  According  to  this  definition,  technology  includes 
not  only  hardware  but  also  the  systemic  and  systematic  processes  associated  with  the 
management  and  solution  of  educational  problems. 

Technology 

The  most  recognized  symbols  of  technological  advancement  are  the  microcomputer 
and  the  Internet  (Office  of  Technology  Assessment,  1988;  Ely,  1993).  The  microcomputer  can 
be  found  in  ninety-eight  percent  (98%)  of  elementary  and  secondary  school  environments 
across  the  United  States  (Quality  Education  Data,  1992)  and  microcomputers  have  become 
so  deeply  ingrained  in  mainstream  society  that  a place  in  the  subculture  of  schooling  is 
assured  (Maddux,  Johnson,  & Harlow,  1993).  “If  we  anticipate  a future  when  more  students 
need  more  learning  there  is  only  one  way  to  meet  that  need  without  diminishing  the  quality 
of  students’  learning  experiences:  we  must  change  the  way  we  deliver  education”  (Twigg, 

1993,  p.  11). 

LeBaron  and  Bragg  (1994),  predict  that  by  the  year  2000  a window  of  opportunity  will 
develop  for  instructional  technology,  when  the  natural  process  of  attrition  will  become 
intense  because  of  an  aging  teacher  workforce.  A new  breed  of  teacher  will  be  able  to  take 
the  lead  in  education.  Ways  for  infusing  teacher  preparation  programs  with  effective  use  of 
instructional  technologies  need  to  be  developed  to  help  this  new  breed  of  teacher  to  better 
serve  the  needs  of  a technological  based  economy. 

LeBaron  and  Bragg  (1994)  state  that  currently,  preservice  teachers  are  graduating 
from  college  with  skills  similar  to  those  of  thirty  years  ago.  With  few  notable  exceptions, 
instructional  technologies  are  typically  presented  to  preservice  teachers  in  a manner  which 
fails  to  model  the  seamless  integration  of  technology  into  the  classroom  (Lampert  & Ball, 
1990;  LeBaron  & Bragg,  1994).  For  instructional  technology  to  truly  become  integrated 
within  the  school  classroom,  appropriate  integration  of  these  technologies  must  be  defined, 
modeled,  and  mentored  for  the  preservice  teacher. 

Systemic  Change 

Those  promoting  systemic  change  propose  three  alternatives  for  bringing  about 
change:  (a)  discard  the  current  system  and  start  over,  (b)  restructure  the  current  system,  and 
(c)  build  a parallel  system.  Robert  Reiser  and  David  Salisbury  (1995)  express  the  view  that 
efforts  to  change  the  current  school  structure  through  a restructuring  effort  involving  pre- 
service teacher  education  are  ineffective.  They  base  this  conclusion  largely  on  the  view  that 


the  "tradition  of  teaching"  inhibits  change  within  the  educational  structure.  The  tradition  of 
teaching"  is  seen  to  have  a powerful  hold  on  teachers  and  makes  it  difficult  for  them  to 
abandon  the  practices  that  have  become  an  integral  part  of  the  public  school  culture. 

However,  Reiser  and  Salisbury  (1995)  assert  the  hope  that  by  modeling  systems  theory  for 
pre-service  teachers,  they  will  see  the  value  of  incorporating  technology  into  their  teaching 
and  will  be  able  to  facilitate  systemic  change  in  education. 

Systemic  change  can  be  facilitated  by  promoting  a mindset  that  includes  strategies 
or  methods  that  are  used  to  understand  education  (Jenlink,  Reigeluth,  Carr,  & Nelson, 

1996).  Systemic  change  “recognizes  the  interrelationships  and  interdependencies  among  the 
parts  of  the  educational  system,  with  a consequence  that  desired  changes  in  one  part  of  the 
system  are  accompanied  by  changes  in  other  parts  that  are  necessary  to  support  those 
desired  changes”  (Jenlink  et  al.,  1996,  p.  22).  The  premise  presented  in  this  paper  is  that 
pre-service  teacher  education,  by  following  these  guidelines,  can  become  an  active  participant 
in  bringing  about  systemic  change  within  the  individual  classroom,  and  also  within  the 
entire  school  system. 

Research  regarding  the  diffusion  of  instructional  technologies  into  the  public  school 
system  indicates  that  the  process  has  been  laboriously  slow  and  in  many  cases  ineffective 
(Cuban,  1986).  However,  education  has  adopted  many  of  these  technologies  and  localized 
change  has  taken  place.  The  process  that  has  facilitated  change  is  similar,  in  many  aspects, 
to  the  processes  that  are  necessary  to  facilitate  systemic  change.  The  process  is  one  of 
mentoring  and  change  facilitation.  A major  challenge  facing  academic  institutions  is  to  train 
teachers  and  staff  to  integrate  technology  within  their  professional  lives  and  within  their 
classrooms.  The  ultimate  goal  for  systemic  change  in  education  is  this  seamless  integration 
of  technology  in  the  classroom. 

The  purpose  of  this  study  was  to  use  the  Concerns-Based  Adoption  Model  (CBAM)  to 
evaluate  the  possibilities  of  facilitating  this  systemic  change  effort  through  entry  level 
teachers  (Teachers  moving  from  the  pre-service  learning  environment  to  first-year  teachers). 
This  study  involved  identifying  pre-service  teachers  concerns  about  integrating  technology 
into  the  classroom  and  their  ability  to  integrate  technology  into  their  instruction,  as  well  as 
the  identification  of  critical  attributes  of  the  effective  integration  of  technology  into  the 
classroom. 

Methodology 

Subjects 

Subjects  consisted  of  27  pre-service  teachers  teaching  in  urban  public  schools  along 
the  Front  Range,  in  Colorado.  The  student  teaching  program  is  designed  to  mentor  the 
development  of  effective  teaching  strategies  and  the  use  of  technology  in  the  classroom  by 
pre-service  teachers.  The  program  utilizes  an  on-site  cooperating  teacher  and  a consultant 
from  the  university  to  mentor  the  student  teacher.  The  mentor’s  work  is  supplemented  by 
seminars  which  the  student  teachers  attend  bi-monthly  during  the  semester.  Further,  the 
university  consultant  visits  the  student  teacher  a minimum  of  six  times  during  the  semester 
and  evaluates  the  student's  teaching  during  a minimum  of  four  of  those  visits.  A 
standardized  form  is  used  to  evaluate  the  student’s  teaching  and  is  designed  to  provide  the 
student  with  constructive  feedback,  both  positive  and  negative. 

In  their  book  Change  in  Schools  (1987),  Gene  Hall  and  Shirley  Hord  state  that 
change  is  a process  not  an  event.  This  is  a process  that  requires  time,  support,  attention  to 
concerns,  facilitation,  and  .a  map  of  the  desired  outcome.  The  components  of  the  Concerns- 
Based  Adoption  Model  (CBAM)  were  designed  by  Hall  and  his  colleagues  at  the  University  of 
Texas.  The  diagnostic  components  of  CBAM  are  designed  to  facilitate  the  change  and 
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adoption  process.  Change  not  only  encompasses  technical  problems  associated  with  the 
adoption  of  an  innovation,  but  also  the  personal  needs  of  the  potential  adopter. 

The  CBAM  model  includes  three  diagnostic  dimensions:  (a)  Stages  of  Concern  (SoC), 
(b)  Levels  of  Use  (LoU),  and  (c)  Innovation  Configuration  Map  (ICM).  Stages  of  Concern 
address  the  intensity  of  the  feelings  and  perceptions  that  the  individual  adopting  the 
technology  is  expressing.  Concerns  are  identified  in  seven  different  areas.  Levels  of  Use 
address  the  behavior  related  to  how  the  individual  uses  the  technology,  with  eight  distinct 
levels  identified.  Specific  behaviors  are  associated  with  each  Level  of  Use.  Finally, 
Innovation  Configuration  Maps  involve  the  development  of  word  maps  that  describe  the 
operational  components  of  an  innovation  and  how  each  can  be  adapted,  or  in  some  cases 
mutated.  In  the  following  sections  each  of  the  diagnostic  procedures  associated  with  CBAM 
are  described  in  greater  detail  and  the  methodology  used  in  this  study  is  explained. 

Stages  of  Concern 

SoC  has  been  used  extensively  in  concern  s research  involving  many  different 
innovation  adoption  situations.  According  to  Hall  and  Hord  (1987)  the  process  of  change  can 
be  more  successful  if  the  ’’concerns"  of  the  individual  are  considered.  The  concept  of 
"concerns"  can  be  defined  as:  “The  composite  representation  of  the  feelings,  preoccupation, 
thought,  and  consideration  given  to  a particular  issue  or  task"  (Hall,  George,  & Rutherford, 
1979,  p.  5).  The  Stages  of  Concern  theory  provides  a means  to  assess  different  perceptions 
toward  systemic  change.  The  SoC  Questionnaire  (SoCQ)  provides  a quantitative  measure  of 
the  intensities  of  the  seven  Stages  of  Concern  dimensions  (see  Figure  1). 
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6 REFOCUSING:  The  focus  is  on  exploration  of  more  universal  benefits  from  the 
innovation,  including  the  possibility  of  major  changes  or  replacement  with  a more 
powerful  alternative.  Individual  has  definite  ideas  about  alternatives  to  the 
proposed  or  existing  form  of  the  innovation. 

5 COLLABORATION:  The  focus  is  on  coordination  and  cooperation  with  others 
regarding  use  of  the  innovation. 

4 * CONSEQUENCE:  Attention  focuses  on  impact  of  the  innovation  on  student  in 

his/her  immediate  sphere  of  influence.  The  focus  is  on  relevance  of  the  innovation  for 
students,  evaluation  of  student  outcomes,  including  performance  and  competencies, 
and  changes  needed  to  increase  student  outcomes. 

3 MANAGEMENT:  Attention  is  focused  on  the  processes  and  tasks  of  using  the 

innovation  and  the  best  use  of  information  and  resources.  Issues  related  to  efficiency, 
organizing,  managing,  scheduling,  and  time  demands  are  utmost. 

2 PERSONAL:  Individual  is  uncertain  about  the  demands  of  the  innovation,  his/her 
inadequacy  to  meet  those  demands,  and  his/her  role  with  the  innovation.  This 
includes  analysis  of  his/her  role  in  relation  to  the  reward  structure  of  the 
organization,  decision  making,  and  consideration  of  potential  conflicts  with  existing 
structures  or  personal  commitment.  Financial  or  status  implications  of  the  program 
for  self  and  colleagues  may  also  be  reflected. 

1 INFORMATIONAL:  A general  awareness  of  the  innovation  and  interest  in  learning 
more  detail  about  it  is  indicated.  The  person  seems  to  be  unworried  about 
himself/herself  in  relation  to  the  innovation;  She/he  is  interested  in  substantive 
aspects  of  the  innovation  in  a selfless  manner  such  as  general  characteristics,  effects, 
and  requirements  for  use. 

0 AWARENESS:  Little  concern  about  or  involvement  with  the  innovation  is  indicated. 


Figure  1.  Stages  of  Concern  About  the  Innovation. 

(Hall  and  Hord,  1987,  p.  60) 

Internal  validity  of  the  SoC  instrument  was  examined  using  Cronbach’s  alpha 
procedure  on  a large  sample  of  data  (N  = 830)  involving  teachers  and  professors  who 
expressed  their  concern  about  an  innovation.  Alpha  coefficients  ranged  from  .64  to  .83,  and 
test-retest  correlations  ranged  from  .65  to  .86.  This  would  indicate  that  the  instrument  had 
internal  consistency  and  stability  for  each  of  the  seven  Stages  of  Concern  (Hall,  George,  & 
Rutherford,  1979,  1986). 

SoC  Methodology 

During  the  course  of  this  study,  SoC  Questionnaires  were  administered  to 
participating  student  teachers  on  three  separate  occasions.  The  questionnaires  focused  on 
the  student  teacher’s  concerns  about  integrating  technology  within  the  classroom. 
Questionnaires  were  administered  during  the  student  teacher’s  regularly  scheduled 
seminars.  The  first  questionnaire  was  used  to  establish  a set  of  baseline  data;  the 
remaining  two  questionnaires  were  used  to  determine  how  their  concerns  changed  during  the 
course  of  their  student  teaching  experience. 
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Innovation  Configuration  Man 

In  their  initial  studies,  Hall  and  his  colleagues  identified  a phenomenon  that  is 
generally  not  acknowledged  when  dealing  with  change  processes.  Users  of  innovations  tend 
to  adapt  and,  in  many  cases,  mutate  innovations.  Although  the  innovation  may  be  identified 
with  the  same  name,  it  may  bear  no  resemblance  to  the  ideals  of  the  developer.  Innovation 
Configuration  Maps  address  this  probable  situation. 

Hall  and  Hord  (1987)  have  developed  special  procedures  for  "mapping”  the  different 
configurations  of  an  innovation.  IC  Mapping  is  innovation  specific.  To  begin  the  IC  Mapping 
process,  materials  must  be  reviewed  and  the  developer(s)  interviewed.  The  primary  task  is 
to  identify  the  operational  components  and  the  behavioral  tasks  associated  with  the 
innovation.  Next  the  variations  of  each  component  need  to  be  identified.  The  overall  goal  of 
IC  Mapping  is  to  develop  a word  map  that  identifies  optimal  conditions  as  well  as 
adaptations  and  mutations  of  the  innovation.  This  results  in  a concept  map  of  the 
innovation. 

ICMs  provide  new  insights  into  the  innovation  and  aspects  for  its  use,  which  help  to 
clarify  the  multiple  forms  that  an  innovation  can  take  and  act  as  a guide  for  facilitating  the 
change  process.  They  provide  word  pictures  of  the  different  ways  in  which  the  innovation  can 
be  used  and  a basis  for  judgments  about  the  effectiveness  of  an  innovation.  In  order  to  bring 
about  systemic  change,  it  is  crucial  to  document  that  "acceptable"  configurations  of  the 
innovation  are  in  use.  The  IC  Map  provides  a device  for  assessing  where  "users"  are  when 
used  in  conjunction  with  Levels  of  Use  (LoU). 

ICM  Me.ttod-Q.lQgE 

In  constructing  the  ICM  a broad  definition  of  technology  was  used  initially.  A 
brainstorming  process  was  used  to  incorporate  the  experiences  of  teachers  currently 
integrating  technology  within  the  classroom.  Teachers  and  professors  were  interviewed 
regarding  their  perceptions  of  technology  use  in  the  classroom.  Concept  maps  were  then 
created  and  merged  with  the  data  obtained  from  the  brainstorming  sessions. 

The  Levels  of  Use  Concept 

Levels  of  Use  provides  a key  ingredient  for  understanding  and  describing 
implementation  of  an  innovation.  Data  collected  from  LoU  interviews  can  provide  useful 
insights  about  staff  development,  evaluation,  planning  and  facilitation  for  leaders  and 
change  facilitators.  Levels  of  Use  gives  the  leadership  and  change  facilitator  a yardstick 
to  monitor  the  rate  of  change.  By  understanding  Levels  of  Use  in  systemic  change,  leaders 
and  facilitators  can  provide  appropriate  interventions  for  each  user  based  on  the  user's 
LoU  category.  According  to  Hall  and  Hord  (1987)  Levels  of  Use  focuses  on  the  behaviors 
that  are  or  are  not  taking  place  in  relation  to  the  innovation  (see  Figure  2).  The  Levels  of 
Use  definitions  presented  in  Figure  3 include  three  "nonuser"  descriptions  and  five  "use" 
descriptions.  When  Hall  and  associates  defined  the  levels,  heavy  emphasis  was  placed  on 
developing  definitions  of  what  could  be  observed  with  each  level  representing  different 
behavioral  approaches.  For  example,  a person  at  Level  of  Use  0,  Nonuse,  is  not  looking  at, 
reading  about,  using,  or  discussing  the  innovation. 
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VI  RENEWAL:  State  in  which  the  user  reevaluates  the  quality  of  use  of  the  innovation, 
seeks  major  modifications  of  or  alternatives  to  present  innovation  to  achieve  increased 
impact  on  clients,  examines  new  developments  in  the  field,  and  explores  new  goals  for  self 
and  the  system. 

V INTEGRATION:  State  in  which  the  user  is  combining  own  efforts  to  use  the  innovation 
with  related  activities  of  colleagues  to  achieve  a collective  impact  on  clients  within  their 
common  sphere  of  influence. 

IVB  REFINEMENT:  State  in  which  the  user  varies  the  use  of  the  innovation  to  increase  the 
impact  on  clients  within  immediate  sphere  of  influence.  Variations  are  based  on  knowledge 
of  both  short-  and  long-term  consequences  for  clients. 

IVA  ROUTINE:  Use  of  the  innovation  is  stabilized.  Few  if  any  changes  are  being  made  in 
ongoing  use.  Little  preparation  or  thought  is  being  given- to  improving  innovation  use  or  its 
consequences. 

Ill  MECHANICAL  USE:  State  in  which  the  user  focuses  most  effort  on  the  short-term, 
day-to-day  use  of  the  innovation  with  little  time  for  reflection.  Changes  in  use  are  made  more 
to  meet  user  needs  than  client  needs.  The  user  is  primarily  engaged  in  a stepwise  attempt 
to  master  the  tasks  required  to  use  the  innovation,  often  resulting  in  disjointed  and 
superficial  use. 

II  PREPARATION:  State  in  which  the  user  is  preparing  for  first  use  of  the  innovation. 

I ORIENTATION:  State  in  which  the  user  has  recently  acquired  or  is  acquiring  information 
about  the  innovation  and/or  has  recently  explored  or  is  exploring  its  value  orientation  and  its 
demands  upon  user  and  user  system. 

0 NONUSE:  State  in  which  the  user  has  little  or  no  knowledge  of  the  innovation,  no 
involvement  with  the  innovation,  and  is  doing  nothing  toward  becoming  involved. 

Figure  2.  Levels  of  Use  of  an  Innovation 
(Hall  and  Hord,  1987,  p.  84) 

Excerpted  from:  The  LoU  Chart:  Operational  Definitions  of  Levels  of  Use  of  the  Innovation 
Austin:  Research  and  Development  Center  for  Teacher  Education,  The  University  of  Texas, 
1975. 


Levels  of  Use  Profile 

Typically  a person  will  move  in  sequence  from  Level  of  Use  0,  Nonuse,  to  Level  of 
Use  IVA,  Routine,  assuming  that  the  innovation  is  appropriate,  the  leader  and  other  change 
facilitators  fulfill  their  roles,  and  time  is  provided.  Activities  associated  with  the 
"nonuser"  range  from  doing  nothing  to  gathering  information  in  preparation  to  use  the 
innovation.  Once  the  user  has  reached  Level  II  and  is  prepared  to  adopt  the  innovation,  they 
are  prepared  to  move  to  the  "user"  categories. 

The  first  use  of  an  innovation  tends  to  be  disjointed  and  erratic.  Most  new  users 
cling  to  the  users  guide  and  concentrate  on  the  day-to-day  uses  more  than  considering  long 
term  uses.  Hall  and  Hord  (1987)  indicate  that  individuals  typically  remain  at  the 
mechanical  level  for  an  extended  period  of  time.  As  the  user  becomes  quite  experienced, 
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they  move  into  Level  of  Use  IVA,  Routine.  Once  a user  reaches  a Routine  Level  of  Use  they 
typically  fall  into  a comfortable  pattern  for  using  the  innovation.  As  users  move  toward  e 
higher  Levels  of  Use  IVB,  Refinement;  V,  Integration;  and  VI,  Renewal,  adaptations  are 
intended  to  improve  the  effectiveness  and  positive  outcomes  from  using  the  innovation. 

The  focus  is  now  on  increasing  effects  with  students. 

Loll  Methodology 

In  this  study,  one  LoU  Interview  was  conducted  with  each  student  teacher  at  the 
conclusion  of  their  student  teaching  experience.  Interviews  were  conducted  at  this  time  to 
determine  their  Level  of  Use  upon  exit.  The  student  teacher’s  Level  of  Use  will,  in  part, 
determine  their  ability  to  facilitate  systemic  change  when  they  enter  the  teaching  profession. 

FIq+q  ArmlvQiQ 


The  data  analysis  procedures  followed  in  this  study  were  a combination  of 
quantitative  and  qualitative  methods.  A comparison  of  means  was  used  to  analyze  the 
scores  obtained  from  the  SoC.  The  LoU  Interviews  were  scored  using  a rubric  created  by  Hall 
and  his  associates  at  the  University  of  Texas.  The  data  obtained  from  individual  student 
teachers  were  compiled  and  comparisons  made  between  student  scores.  Trends  were 

identified  and  outliers  explored  in  greater  depth. 

Results 

SoC 


For  students  entering  the  student  teaching  experience,  their  Self  concerns 
(awareness,  informational,  and  personal)  were  high  (see  Series  1 of  Figure  3).  Task  and 
Impact  concerns  were  less  evident.  Previous  research  (Hall  & Hord,  1987),  indicates  that 
this  should  be  expected.  “Concerns  at  this  point  have  to  do  with  feelings  of  potential 
inadequacy,  self-doubts  about  the  knowledge  required,  or  uncertainty  about  the  situation 
they  are  about  to  face.  Typical  statements  reflecting  these  types  of  concerns  are:  I wonder  if 
I know  enough  to  teach  them.’  ‘Will  I be  able  to  control  them?”’  (Hall  & Hord,  1987,  p.  57). 

The  results  of  the  second  SoC  indicated  higher  Task  concerns  (management).  This 
instrument  was  administered  midway  through  the  semester  (see  Series  2 of  Figure  3).  As 
student  teachers  transition  into  the  role  of  teacher,  concerns  regarding:  logistics,  classroom 
management,  and  preparation  are  expected  to  increase  (Hall  & Hord,  1987).  As  students 
begin  to  observe  the  new  role  of  the  student  teacher,  they  place  added  demands  on  the 
student  teacher’s  time  and  on  their  ability  to  manage  the  classroom. 

As  the  student  teachers  progressed  through  their  student  teaching  experience, 

Impact  concerns  (consequence,  collaboration  and  refocusing)  increased  in  intensity  (see  Series 
3 of  Figure  3).  “Ultimately,  teachers  can  become  predominately  concerned  about  how  their 
teaching  is  affecting  students  and  about  how  they  can  improve  themselves  as  teachers”  (Hall 
& Hord,  1987,  p.  57).  At  the  conclusion  of  the  experience  their  overall  concerns  regarding 
instructional  technology  were  all  intensified. 


Stage  0 Stage  1 Stage  2 Stage  3 Stage  4 Stage  5 Stage  6 

Awareness  Informational  Personal  Management  Consequence  Collaboration  Refocusing 

Stages  cf  Concern 

KM 

Initially  the  ICM  was  developed  to  identify  both  effective  teaching  strategies  and 
technology  in  the  classroom.  Each  map  described  the  optimal  and  desired  behaviors  that  the 
teacher  was  to  exhibit  if  the  technology  was  used  at  Loll  IVA,  Routine.  During  the  course  of 
the  initial  LoU  interviews,  the  ICM  had  to  be  redesigned  to  look  at  hard  technologies 
instead  of  a more  broad  view  of  technology.  The  final  version  of  the  ICM  included  a checklist 
of  technologies  designed  to  assess  the  availability  of  specific  technologies  in  the  classroom  as 
well  as  a word  map  that  behaviorally  defined  the  use  of  those  technologies. 

LoU 

Upon  exiting  their  student  teaching  experience,  the  student  teachers’  LoUs  ranged 
from  Level  f through  Level  IVA.  The  percentage  of  nonusers  were  20%.  while  the  users  were 
80%.  However,  the  heaviest  grouping  of  users  were  Level  III,  Mechanical,  at  54%.  The 
remaining  student  teachers  were  at  Routine  LoU.  Only  26%  of  the  student  teachers  are 
positioned  as  entry  level  teachers  to  facilitate  systemic  change. 

Discussion 

Without  a doubt,  technology  will  continue  to  advance  and  people  must  be  supported 
in  the  adoption  process.  Educators  must  harness  the  power  of  technology  to  aid  and  assist 
their  instructional  endeavors.  Only  when  technology  has  advanced  to  become  an  integral 
part  of  the  teacher’s  instructional  repertoire  will  we  see  the  advantages  that  technology  can 
provide.  For  technology  to  become  an  integral  component  of  the  educational  classroom 
environment  and  learning  process  of  our  schools,  teacher  preparation  programs  and  school 
districts  must  provide  the  necessary  support  to  help  teachers  to  develop  the  skills  essential 
to  prepare  their  students  to  compete  in  a technologically  based  society.  There  are  two 
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aspects  of  support  that  need  to  be  considered:  technical  support  - aimed  at  establishing 
and  maintaining  connectivity  - as  well  as  curriculum  support.  Curriculum  support  would 
provide  for  teaching  materials,  documentation  for  teachers  and  students,  location  of  sources 
of  potential  benefit;  suggestions  about  integrating  resources  into  the  curriculum,  and 
inservice  training  aimed  at  facilitating  integration.  Teacher  education  programs  and  school 
districts  that  implement  or  emphasize  only  one  type  of  support  are  not  likely  to  succeed 
(Maddux,  1994).  This  is  a process  that  should  begin  with  student  teaching  and  extend 
through  the  first' few  years  of  actual  teaching. 

It  has  been  argued  that  students  leave  college  and  adopt  the  methods  of  the 
institution  at  which  they  are  hired,  forgetting  what  they  learned  in  college.  They  leave  college 
using  the  skills  they  have  learned  at  a mechanical  level  of  use.  This  level  of  use  is  easily 
deteriorated  (Hall  & Hord,  in  press).  Mechanical  level  users  fall  back  on  their  most 
comfortable  method.  Unfortunately  these  methods  are  not  always  the  most  productive.  To 
ensure  that  novices  progress  beyond  a mechanical  level  of  use,  a procedure  for  mentoring 
them  in  the  continued  use  of  instructional  technologies  needs  to  be  instituted. 

Currently,  education  is  not  meeting  the  needs  of  society  (Fad  & Ryser,  1993; 
Superintendents,  1988;  LeBaron  & Bragg,  1994;  Wirth-bond,  Coyne,  & Adams,  1991). 

There  is  a lack  of  technological  integration  within  teacher  education  today.  Teacher 
education  programs  need  to  provide  instruction  for  teachers  that  facilitates  the  integration  of 
computing  technologies  into  the  classroom  and  their  professional  lives.  A routine  level  of  use 
better  ensures  the  integration  of  technology  in  the  classroom  (Hall  & Hord,  in  press).  To 
achieve  this  end,  a plan  for  viewing  how  technology  is  integrated  within  the  classroom  needs 
to  be  developed.  This  study  has  presented  a perspective  for  facilitating  systemic  change  by 
way  of  integrating  technology  within  the  teacher  preparation  program.  The  study  also 
elicited  valuable  information  for  developing  specific  interventions  to  be  used  with  pre-service 
teachers.  The  components  of  CBAM  can  be  used  to  bring  about  systemic  change  in 
education.  Concerns  and  use  data  and  the  prescribed  interventions  along  with  the  mentoring 
of  pre-service  educators  can  provide  a foundation  for  a ground-up  approach  to  effect  systemic 
change  in  education. 

In  their  efforts  to  bring  about  systemic  change  in  education,  educators  and 
instructional  designers  need  to  use  technology  to  assist  in  instructional  management  and 
improvement,  communication  with  both  peers  and  experts  in  their  fields,  and  identification 
of  new  instructional  methods  and  resources.  These  tools  will  help  teacher  s create  instruction 
that  promotes  self- regulatory  learning  behaviors  such  as  coping  strategies,  problem-solving 
and  decision  making  skills,  goal  setting,  and  planning. 

Future  Research 

Student  teachers  have  provided  a baseline  group  from  which  to  launch  further  study. 
This  baseline  group  has  provided  the  beginnings  of  a longitudinal  study  regarding  the 
student  teacher’s  adoption  and  retention  of  the  technologies  included  in  this  study.  Future 
studies  need  to  look  at  the  Levels  of  Use  at  the  cooperating  teacher  level  and  include  a LoU 
interview  at  the  beginning  of  the  student  teacher  experience.  This  will  allow  researchers  to 
take  into  account  the  mentoring  process  that  occurs  during  the  student  teaching  experience. 
Other  avenues  to  consider  include:  (a)  a longitudinal  study  designed  to  follow  student 
teachers  through  their  first  few  years  in  the  classroom,  (b)  university  faculty  and  their  use  of 
technology  in  the  classroom,  as  well  as  in  their  professional  lives,  and  (c)  school  district 
efforts  to  integrate  technology. 

Systemic  change  is  a process  that  requires  time,  collaboration,  flexible  leadership, 
and  facilitators  who  understand  all  aspects  of  the  change  process.  From  the  research 
conducted  in  this  study  there  is  an  indication  that  in  order  to  facilitate  systemic  change, 
student  teachers  need  to  be  better  prepared  to  enter  the  classroom.  To  achieve  this  end, 
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CBAM  should  be  used  as  a tool' to  assess  and  facilitate  the  progress  of  student  teachers  in 
reaching  a routine  LoU  in  the  integration  of  technology  within  the  classroom.  The  theory, 
research,  and  instruments  offered  by  CBAM  provide  a valuable  tool  for  eva.:  ating  and 
facilitating  systemic  change. 
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I.  Introduction 

The  growth  of  the  information  industry  and  new  computer  technologies  enables  us  actual  use  in  the 
field  of  education  and  training.  Among  these  high  technologies,  educators  have  concentrated  on  multimedia 
and  telecommunications,  developing  practical  programs  for  instruction.  Multimedia  is  now  recognized  as  a 
basic  technology  for  developing  and  using  educational  software.  Our  attention  has  moved  from  stand-alone 
computer  systems  to  integrated  systems  which  include  CD-ROM  and  interactive  video.  Furthermore  personal 
desktop  multimedia  systems  can  be  linked  with  other  systems  through  computer  networks,  enabling  free 
exchange  of  various  types  of  information  such  as  text,  images,  and  voice.  Interpersonal  computing  is 
beginning  to  replace  personal  computing. 

However,  most  discussion  and  development  relating  to  the  above  technologies  emphasize  their 
mechanical  capabilities  rather  than  their  educational  potential.  Technology  alone  cannot  provide  the 
answers  when  we  deal  with  human  activities  such  as  learning  and  instruction.  'The  task,  the  culture,  the 
social  structure,  and  the  individual  human  are  all  essential  components  of  the  job,  and  unless  the 
computational  tools  fit  'seamlessly'  within  this  structure,  the  results  will  be  fai lure' (Norman,  1991).  In 
order  to  derive  educational  improvement  from  the  technologies,  we  need  to  establish  and  use  a sound 
conceptual  and  technical  framework  for  the  development  of  educational  programs.  It  is  especially  important 
in  relation  to  the  super  high-speed  information  network  being  constructed  by  the  Korean  government.  Unless 
we  establish  theoretical  background  and  integrate  fundamental  and  common  technologies  for  educational 
applications,  it  will  be  difficult  to  enhance  the  quality  and  productivity  of  educational  programs  and 
ultimatel)  to  improve  public  welfare. 

This  study  identifies  the  basic  technologies  necessary  for  distance  education  and  specifies  their 
content  ard  scope.  The  term,  “distance  education”,  in  this  paper  means  high-speed  network-bused 
interactive  instruction  and  learning  systems  and  is  different  from  the  existing  distance  education  systems 
in  many  aspects.  The  characteristics  of  the  existing  systems  are  that  instruction  and  learning  activities 
take  place  with  pre-made  printed  materials,  broadcasting,  and  audio  and  video  tapes.  The  1 imi  tation  of 
these  systems  is  that  the  learner  has  no  personal  contact  with  other  students  and  that  it  is  difficult  for 
teachers  end  students  to  exchange  taeir  opinions  and  feelings.  Since  education  is  primarily  performed  by 
interactions  between  students  end  teachers,  we  need  to  increase  the  opportunities  for  educat  onal 
interaction.  In  fact,  instructors  have  made  an  effort  to  link  the  learners  and  instructors  by  applying 
communication  techniques  such  as  facsimile,  computer -mediated  communication  and  video  and  audio 
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conferencing  to  the  learning  process.  Among  mediated  two-way  communication  systems,  high-speed  information 
network  or  B- 1 SDN  (Broadband  Integrated  Service  Data  Network)  is  expected  to  be  the  most  powerful  one. 
Therefore,  to  maximize  the  effectiveness  of  high-speed  networks  in  education,  we  need  to  establish 
conceptual  backgrounds  and  develop  fundamental  and  common  techniques  for  designing  and  implementing 
distance  education  systems.  Otherwise,  we  will  repeat  the  same  mistakes  as  we  have  done  with  the  previous 
technolog).  When  we  have  the  theoretical  and  technical  infrastructure,  its  usefulness  will  be  enormous:i 
will  be  used  every  time  an  application  system  is  developed  in  various  fields  of  education  and  training  in 
schools,  industry,  the  army,  hospitals,  and  the  home. 


II.  Basic  Concepts  of  Distance  Education 
A.  Definition 

Distance  education  is  not  a new  concept,  but  one  which  developed  from  the  18th  century  when  a 
stenography  lesson  was  offered  by  mail  in  Boston.  Distance  education  is  an  instruction  delivery  system 
mediated  by  print,  audio  and  video  materials,  and  broadcasting  without  face-to-face  interacton  between 
instructors  and  students.  The  characteristics  of  distance  education  include  an  absence  of  personal 
interaction,  dependence  on  individualized  learning  materials,  two-way  communication  through  various  mediu, 
a large  number  of  students  and  their  individualized  learning,  and  it  own  supporting  system  which  is 
different  from  conventional  systems  (Education  Ministry  of  Korea,  1994). 

Distance  education  has  developed  as  new  communication  technology  evolves.  It  is  divided  into  3 
generations  based  on  the  technology  used.  The  first  generation  was  based  on  mail,  the  second  on  mass 
communication  such  as  broadcasting.  The  third,  interactive  distance  education,  has  electronic 
telecommunication  technology  as  its  basis.  This  paper  focuses  on  the  third  generation  and  discusses  its 
advantages  and  disadvantages. 

The  third  generation  distance  education  systems  are  also  divided  into  several  types  depending  on 
underlying  telecommunication  technology.  The  most  economical  and  most  quickly  imp lemen table  type  is  based 
on  PSTN (Public  Switched  Telephone  Network).  It  provides  very  primitive  asynchronous  tutorials  in  which  the 
students  download  instructional  contents  to  their  computers,  study  them,  and  ask  questions  to  teachers  o 
program  providers  by  using  e-mail  and  bulletin  boards.  It  is  a very  popular  form  of  distance  education 
internationally.  The  tyues  of  distance  education  systems  which  draw  our  attention  most  today  are 
interact i % e video-based  distance  lectures  and  computer-based  multimedia  distance  education  systems.  This 
paper  discusses  these  two  types  of  distance  education  in  more  detail. 


B.  Distance  education  based  on  conferencing  technology 
1.  V idee -Conference-Based  Distance  Education 

Interactive  video-based  distance  lectures  are  based  on  existing  analog  cable  TV  broadcasting 
technolog).  The  basic  hardware  includes  a CODEC  which  converts  anau;g  audio  and  video  signals  into  digit;. 1 
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ones,  video  cameras,  TV  monitors  and  audio  sets.  ISDN,  satellite,,  and  coaxial  and  optical  cables  are  used 
for  communication.  Figure  1 shows  the  the  structure  of  general  systems.  This  system  requires  very 
expensive  hardware  and  equipment,  but  not  complex  software.  It  has  a short  construction  time  which  count:; 
for  its  international  popularity. 


Figure  1.  Configuration  of  General  Distance  Video-lecture 


From  the  educational  point  of  view,  the  distance  lecture  essentially  shares  the  advantages  and 
disadvantages  of  the  conventional  lecture  where  an  instructor  teaches  many  students  at  a specific  time  and 
space.  The  teaching  and  presentation  skills  of  the  instructor (including  interaction  strategies  with 
students)  affect  the  successfulness  of  the  system,  even  more  so  since  the  instructor  may  accommodate  more 
students  than  in  the  conventional  setting.  The  instructor  should  also  be  able  to  operate  the  related 
broadcasting  and  computer  equipment  and  software.  The  superiority  of  distance  lectures  over  the 
conventional  ones  isjthat  it  can  accommodate  more  learners  in  multiple  classrooms  that  are  separated 
geographically,  yet  all  students  still  have  the  opportunity  to  ask  realtime  questions  to  the  teacher.  Th  s 
advantage  is  especially  important  when  qualified  instructors  are  insufficient. 

However,  individual  students  still  need  to  leave  their  working  site  or  home  and  go  to  the  nearest 
distance  education  classroom.  They  cannot  completely  be  free  from  time  and  space  considerations. 
Additionally,  interactions  between  the  teacher  and  student  are  very  limited.  First  of  all,  students  have 
fewer  charccs  for  interaction  and  it  is  difficult  to  provide  various  educational  interactions  such  as  that 
between  learners.  Finally,  it  cannot  promise  the  effectiveness  and  efficiency  which  can  be  provided  by 
individualized  learning  systems  since  it  is  still  a group  learning  system  based  on  UN  interaction. 


2.  Computer-Based  Multimedia  Distance  Education  System 
The  computer-based  multimedia  distance  education  system  can  be  offered  as  a solution  for  most  problens 
of  the  viceo-conference-based  systems.  This  system  is  an  educational  application  of  computer-  or  desktop-- 
conferencing  technology.  Its  basic  configuration  has  the  teachers’  and  students’  individual  computers 

i inked  vie  high-speed  networks.  Through  the  network,  they  can  also  access  digital  libraries  and 
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educational  databases  containing  numerous  learning  materials  and  programs.  Teaching  can  be  in  various 
instructional  forms  such  as  individualized  learning  by  accessing  the  appropriate  materials  stored  in 
educational  databases,  virtual  attendance  at  experts'  lectures,  and  collaborative  learning  by  discussion 
between  learners  in  different  places.  Now  students  are  completely  free  from  the  time  and  space 
considerations  to  learn  materials  and  develop  necessary  skills.  It  is  possible  for  teachers  or  course 
developers  to  collaborate  to  create  quality  educational  programs  efficiently.  As  they  can  can  provide 
natural,  individualized,  and  flexible  educational  interactions,  they  can  be  expected  to  be  the  most  wide  y 
used  instructional  delivery  system  along  with  the  use  of  high  speed  computer  network  technology. 

However,  computer-based  distance  learning  systems  cannot  be  implemented  simply  by  build:ng  the 
physical  infrastructure  such  as  network  and  broadcasting  equipment.  They  require  software  technology  to 
support  various  learner  and  teacher  interactions,  process  multimedia  effectively,  and  link  to  databases. 
Moreover,  the  theoretical  backgrounds  and  design  principles  to  maximize  the  potential  of  this  new  flexib  e 
educational  applications  are  not  currently  available.  Therefore,  it  will  take  considerable  time  to 
implement  computer-based  multimedia  distance  education  systems  based  on  a generalized  and  sound  conceptual 
and  technical  infrastructure.  Although  there  are  few  example  of  its  application  in  Korea,  its  ^ducat iona 
potential  has  been  recognized  along  with  the  construction  of  Information  Super  Highway  in  Korea. 

i 


III.  The  Current  Status  of  Distance  Education  Using  Conferencing  Technology  in  Korea 

In  Korea,  it  is  popular  to  implement  interactive  video-based  distance  education  systems.  Although 
there  is  little  technical  problems,  the  hardware  is  expensive.  A representative  case  of  the  systems  is 
that  of  Hongchun,  a sparsely  populated  city  in  Korea.  This  system  connects  5 elementary  schools  in 
Hongchun.  The  major  purpose  of  the  system  is  to  solve  the  problems  of  multi-lesson  in  which  a teacher 
deals  with  2 or  3 different  grades  in  a single  classroom.  Figure  2 shows  the  overall  configuration  of  th  s 
system. 

The  goverment  selected  Korea  Open  University  as  a model  distance  college  education  inst'tute.  It 
applies  ar.  interactive  video-based  distance  education  system  to  deliver  lectures  located  in  one  learning 
center  to  students  who  are  separated  from  the  instructor  geographically.  Figure  3 shows  the  overall 
configuration  of  this  system.  The  purpose  of  the  system  is  to  provide  more  student-oriented  and 
interact i\e  distance  education  programs,  to  improve  the  quality  of  education  services,  to  investigate  the 
use  of  information  super  highway  in  the  area  of  education,  and  to  increase  the  educational  opportunities 
for  the  public. 

Many  other  cases  of  interactive  video-based  distance  education  in  Korea  are  related  to  col'ege  education 
and  are  summarized  in  table  1.  Many  large  companies  in  Korea  are  also  using  or  planning  to  use  distance 
education  systems  to  improve  the  quality  and  quantity  of  employee  or  customer  training  programs. 
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Figure  2.  System  Configuration  of  Distance  Education  Testbed 
Linking  Elementary  Schools  at  Hongchun 
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Figure  3.  System  Configuration  of  Korea  Open  University 


34y 

F-46 


Name  of  Inst i tute 

Start ing 
Date 

Application  Area/ 
Purpose 

System  Configuration 

Seoul  National 
University 

5/94 

Course  offering  in 
liberal  arts  & 
sciences 

Seoul  National  Univ. -Internat ional  Studio 
-R&D  Institute 

Yonsei  University 

12/94 

MBA  course  offering 

Yonsei  Uni.  Management  Gradute  School- 
Sambo  Computer  Inc. 

Suwon  University 

9/94 

International  course 
offering 

Suwon  Univ. -Univ.  of  Utah 

Elementary  School 
in  Hongchun 

3/95 

Solution  to  Multi- 
lesson 

5 Elementary  Schools  at  Hongchun 

Korea  Open 
University 

11/95 

Replacement  to 
at  tendace-requi red 
courses 

Korea  Open  Uni.  main  campus-13  local 
learning  centers 

Samsung  Group 

6/96 

Technology 
education,  HRD, 
Public  Service 

Samsung  Group-High-quality  Universities 
in  Korea-Foreign  Experts 

Daewoo  Group 

4/95 

MBA  course  offering 

Daewoo  Group-Uni v.  of  Michigan 

Table  1.  Representative  Cases  of  Interactive  Video-Based  Distance  Education  in  Korea 


There  are  few  cases  of  computer-conference  based  distance  education  in  Korea.  However,  along  with  the 
construction  of  the  information  super  highway  in  Korea,  its  educational  potential  has  been  recognized  and 
its  underlying  technologies  have  been  developing.  Among  those  technologies,  there  is  DOORAE (Distributed 
Object  Collaborative  Environment  for  Interactive  Multimedia  Service)  developed  by  Hwang  (Hwang,  1995).  It 
is  an  educational  application  of  computer-conferencing  technology  and  a software  platform  to  implement 
distance  education  systems.  Its  outstanding  capability  is  what  allows  various  interactive  co‘ laborat ion 
between  teachers  and  students.  This  platform  has  been  applied  to  constructing  the  educational 
infrastructure  for  Sungkyunkwan  University.  In  order  to  be  a complete  tool  for  implementing  distance 
education  systems,  the  following  functions  should  be  added:  interfacing  with  educational  databases,  more 
sophisticated  multimedia  data  editing,  and  application  sharing. 

Most  distance  education  systems  developed  so  far  concern  only  the  construction  of  hardware  systems  and 
the  development  of  application  systems  that  will  be  directly  accessed  by  end  users.  However,  along  with 
this  effort  we  should  also  develop  basic  and  common  technologies  which  will  be  used  whenever  we  develop 
application  systems.  This  approach  will  allow  us  to  exclude  the  redundant  investment  in  deve’oping 
underlying  technologies  for  each  application  and  to  enhance  the  productivity  and  quality  of  application 
systems. 
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IV.  The  Goal  of  Technological  Framework  for  Network-based  Distance  Education 


A.  The  Plan  for  Constructing  B-ISDN  in  Korea 

To  set  the  goal  of  technological  foundation  for  distance  education,  it  is  necessary  to  understand  the 
plan  for  building  B-ISDN  in  Korea.  Table  2 shows  the  construction  plan  summary  for  the  information  super 
highway  ir.  Korea.  Data  transmission  rates  will  steadily  increase  from  155Mbps  to  lOOGbps.  Service  areas 
will  expand  from  business  areas  to  the  general  public. 


Phase  1 
(1995  - 1997) 

Phase  2 
(1998  - 2002) 

Phase  3 
(2003  - 2007) 

Phase  4 
(2008  - 2015) 

Baud  Rate 

155  - 622  Mbps 

2. 5 - 10  Gbps 

100  Gbps 

More  than  100 
Gbps 

Service  Areas 

Business  areas 

Extended  business 
areas 

* 

Public  residence 
areas 

Extended  public 
residence  areas 

i 

Table  2.  Construction  Plan  Summary  for  the  Information  Super  Highway  in  Korea 


B.  The  Characteristics  of  Technological  Framework  for  Computer-Based  Multimedia  Distance  Education 

Network-based  distance  education  systems  are  based  upon  the  integration  of  technologies  related  to 
education  and  computer  networks.  Because  it  is  a delivery  strategy  of  education,  it  should  be  based  on 
educai  on- specific  technology  such  as  theories  of  learning  and  instruction  and  general  princ:ple  and 
procedures  for  designing  and  implementing  educational  programs.  Also,  since  it  is  operated  on  a computer 
network,  it  is  necessary  to  use  fundamental  network-dependent  computer  technology  about  user- interface, 
information-processing,  database  systems,  and  networking.  Figure  4 shows  the  technologies  necessary  for 
network- based  education. 
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♦ School  education  ♦ Industry  training 

♦ Continuing  education  ♦ Mlitaiy  training 

♦ Ictanational  education  ♦Msdical  training 
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Figure  4.  Technologies  Related  to  Computer-Based  Multimedia  Distance  Education 


C.  The  Goals  of  Network-based  Education 

The  goals  of  developing  a conceptual  and  technical  foundation  for  distance  education  differ  for  each 
stage  of  construction  plan.  The  first  phase (1995-1997) is  intended  to  design  and  implement  distance 
education  based  on  multimedia  database  systems  and  to  accomplish  distributed  individualized  earning. 
Figure  5 shows  the  configuration  of  the  first  phase.  Learners  can  share  educational  information  and  there 
will  be  limited  communication  between  learners  and  instructors  through  electronic-mail. 
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Figure  5.  Multimedia  Database  System-Based  Distance  Education 
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The  goal  of  the  second  period (1998-2002)  is  to  design  and  implement  distance  education  based  on  real 
time  interpersonal  computing  and  to  accomplish  distributed  cooperative  learning.  Instructors  can  easily 
access  numerous  multimedia  database  systems  through  their  global  and  local  managers  and  freely  communica  e 
with  each  other.  Also  it  is  possible  to  group  users  in  various  ways,  such  as  student -con trolled  grouping, 
teacher-controlled  grouping  and  system-controlled  grouping.  Figure  6 shows  the  configuration  of  the  second 
phase. 


Figure  6.  Real-time  Interpersonal  Computing  for  Distance  Education 

The  third  phase (2003-2007) is  intended  to  design  and  implement  distance  education  based  on  intelligent 
network  ard  to  accomplish  distributed  adaptive  learning  in  which  instruction  is  provided  based  upon  a 
user’s  history,  need,  and  personal  characteristics  such  as  aptitude  and  personality.  The  intelligent 
interface  brings  user-friendly  interactions  between  learners  and  systems  and  even  extends  educational 
services  to  disabled  learners.  Figure  7 shows  the  third  phase  configuration. 


Figure  7.  Configuration  of  Intelligent  Distance  Education 
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The  goal  of  the  fourth  phase (2003-2007) is  to  design  and  implement  distance  education  using  virtual 
technology  and  to  accomplish  distributed  realistic  learning  which  appeals  to  all  human-senses.  It  is 
possible  to  represent  virtual  classrooms  and  virtual  schools.  Figure  8 shows  the  configuration  of  the 
fourth  phase. 


Student  Group 


Intelligent  Network  Layer 


Figure  8.  Virtual  Reality-Based  Distance  Education 


V.  The  Content  and  Scope  of  the  Component  Technologies  Supporting  Distance  Education 
A.  Educational  Service  Modeling 

Technology  supporting  distance  education  consists  of  4 parts:  modeling  of  educational  services, 
courseware  authoring,  structuring  and  storing  educational  information,  and  educational  user-  nterface.  ' 
Modeling  educational  services'  includes  the  development  of  theories  and  guidelines  necessary  to  design  and 
develop  various  network-based  education  systems.  'Courseware  authoring'  technology  allows  users  to  design 
and  develop  various  types  of  network-based  courseware  without  expert  knowledge  or  skills  on  networking, 
programming  and  instructional  des ign. 'Structuring  and  storing  educational  information'  technology  includes 
what  is  necessary  to  structure  educational  materials  based  on  their  characteristics  and  to  store  them  as 
an  integrated  educational  database  system.  This  technology  assures  the  efficient  retrieval  of  educationa 
materials.  'Educational  user- interface'  technology  helps  users  to  interact  with  the  system  easily  in 
retrieving  and  using  educational  materials  stored  in  the  database  systems.  Figure  9 shows  the*  4 componen 
technologies  supporting  distance  education 
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Figure  9.  Component  Technologies  Supporting  Distance  Education 


B.  The  Content  and  Scopes  of  Component  Technologies  for  Distance  Education 


1.  Educational  Service  Modeling 

Network-based  educational  service  modeling  is  the  groundwork  for  developing  the  other  three  component 
for  network-based  education.  In  modeling  the  services,  the  concept  of  interpersonal  computing  should  be 
emphasized.  Interactivity  is  one  of  the  most  important  instructional  and  learning  factors  in  all 
educational  and  training  programs.  Educational  interactions  take  place  between  learners  and  instructors, 
between  peer  learners,  and  between  learners  and  learning  materials.  The  interactions  between  learners  and 
learning  materials  have  been  emphasized  in  the  existing  distance  education  systems  using  courseware  and 
only  limited  interactions  between  instructors  and  learners  have  been  possible.  As  telecommunication 
technology  continues  to  evolve,  interactions  between  learners  and  between  learners  and  instructors  are 
possible,  thereby  enhancing  the  effectiveness  of  distance  education. 

In  modeling  educational  services,  it  is  very  important  to  develop  the  techniques  necessary  to  manage 
the  types  and  direction  of  educational  interaction  according  to  the  network  construction  plan  and  the 
available  network-related  technology.  The  designing  principles  and  guidelines  for  various  network-based 
instructional  strategies  such  as  distributed  individualized  and  collaborative  learning  need  to  be 
developed.  By  establishing  and  integrating  models  of  subject  matters  and  learner  characteristics,  the 
methodology  for  designing  and  implementing  distributed  intelligent  learning  systems  should  be  provided.’ 

The  specification  of  educational  environments  which  can  be  enhanced  by  applying  virtual  reality 
technologies  should  be  presented  and  the  designing  principles  and  guidelines  for  creating  and  implementing 
virtual  classrooms  are  expected  to  be  provided.  In  modeling  the  services,  we  must  always  consider  the 
three  aspects  of  educat ion  (learning,  teaching  and  authoring).  All  three  principles  and  guide* ines  should 
be  stored  in  a database  systems  so  that  everyone  is  able  to  reference  them  readily. 
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2.  Courseware  Authoring  Technology 


In  order  to  improve  the  quality  and  productivity  of  network-based  courseware,  authoring  technology 
should  be  developed  and  provided  for  users  as  an  easy-to-use  software  tool.  This  technology  (ran  be 
developed  by  applying  educational  needs  to  fundamental  software  technologies  related  to  mult:media  data 
processing,  object-oriented  databases,  artificial  intelligent  including  expert  systems*  and  multiuser 
interfaces. 

To  specify  the  content  and  scope  of  courseware  authoring  technology,  it  is  first  important  to  provide 
the  utilities  for  interfacing  with  educational  multimedia  database  and  distributed  learning  management 
systems.  By  developing  the  MHEG (Mul t imedia  and  Hypermedia  Expert  Group)  engine,  we  need  to  improve  the 
capabilities  of  processing  multimedia  information.  Also  a graphical  user- interface,  providing  a meaningful 
and  easy-to-use  for  both  courseware  authors  and  learners,  should  be  developed.  The  authoring  utilities  for 
supporting  various  educational  interactions  are  important  and  it  is  necessary  to  provide  the  multiuser- 
interface  for  collaborative  works  such  as  distance  lecture,  cooperative  learning,  and  collaborative 
authoring.  It  is  important  to  support  the  development  of  intelligent  tutoring  systems  in  distributed 
environments.  It  is  also  necessary  to  provide  the  engines  for  storing  expert  modules  contain*ng  domain 
knowledge  and  problem-solving  rules,  learning  strategy  module  containing  instructional  strategy  and 
diagnosis  and  prescription  rules,  and  learner  model  modules  containing  each  individual’s  performance 
history,  aptitude,  and  personality  factors.  It  is  also  helpful  to  provide  a natural  language  based  user 
interface  for  both  authors  and  learners.  When  virtual  reality  technology  is  readily  available,  it  is 
important  to  develop  a human  sense-based  user- interface  for  both  authors  and  learners  and  to  provide  the 
utilities  for  activating  virtual  reality  equipment,  delivering  telepresence,  editing  and  rendering  real- 
time animation,  and  implementing  holograms. 


3.  Technology  to  Store  and  Structure  Educational  Information  and  Materials 

When  using  stand-alone  computer-based  education  systems,  we  can  use  courseware  without  any 
intervention  once  it  is  developed.  However,  once  educational  material  is  developed  in  a network-based 
environment,  it  is  stored  in  a database  and  then  retrieved  when  needed.  Therefore  educationa’  materials 
should  be  systematically  structured  and  stored  so  that  the  correct  material  for  the  user’s  needs  and 
characteristics  can  be  easily  retrieved. 

An  index  engine  for  educational  multimedia  and  hypermedia  information  and  an  automatic  analyzer  and 
classifier  of  the  internal  and  external  structure  of  information  needs  to  be  developed.  The  advanced 
synchronization  model  to  present  multimedia  information  in  various  ways  in  learning  processes  needs  to  be 
provided.  To  support  instructional  activities  for  teachers,  navigation  and  browsing  tools  should  be 
provided.  In  order  for  educational  information  providers  to  revise  and  update  educational  materials,  a 
version  menager  is  required.  It  is  also  necessary  to  model  educational  multimedia  databases  and  to  develop 
navigation  techniques  in  hypermedia  systems.  Real-time  interpersonal  educational  information,  should  be 
based  on  the  same  properties  as  those  for  the  multimedia  and  hypermedia  information  should  be  developed 
based  upor.  its  properties.  To  link  distributed  multimedia  database  systems  efficiently,  loca'  and  global 
managers  must  be  developed.  For  the  intelligent  and  virtually  realistic  information,  the  same  techniques 
mentioned  above  need  to  be  developed  based  upon  its  characteristics  and  requirements. 

I 


4.  Educational  User- interface  Technology 


Since  most  learners  and  instructors  have  no  expert  knowledge  or  computer  or  telecommunication  skill:;, 
we  need  to  develop  and  provide  an  easy-to-use  and  friendly  user- interface.  Input  and  output  techniques  for 
multimedia  data  processing,  media  conversion  techniques,  and  a protocol  allowing  multiple-users  are 
necessary  in  developing  the  educational  user- interface. 

To  specify  the  content  and  scope  of  the  technology  about  educational  user- interface,  it  is  first 
important  to  provide  two  and  three  dimensional  graphical  user- interface  supporting  educational  activities 
for  non-expert  users,  the  metaphors  modeling  various  educational  interactions,  and  the  cognitive  user- 
interface.  It  is  also  necessary  to  develop  various  user-grouping  techniques  such  as  learner-controlled 
grouping  based  on  users  needs  and  system-controlled  grouping  based  on  users’  performance  history, 
aptitude,  and  personality  factors.  An  interface  supporting  educational  conversation  and  group  discussion 
in  cooperative  learning  is  required.  It  is  also  necessary  to  develop  an  intelligent  agent  that  models  the 
learner  ard  selects  the  right  learning  tasks  for  him  or  her.  An  image  agent  needs  to  be  developed  to 
direct  anc.  advise  educational  activities.  We  must  develop  educational  natural  language-based  interfaces, 
which  generate  and  interpret  various  inputs  and  outputs  including  voice,  pattern,  image,  motion,  gesture, 
and  vision.  Finally,  for  the  databases  of  the  techniques  about  real-time  simulation  and  telepresence  we 
need  to  develop  an  interface  that  guides  users  to  create  the  setting  of  virtual  collaboration  without 
space  and  time  constraints. 


VI.  Conclusion 


To  maximize  the  use  of  technology  in  education  and  to  achieve  educational  improvement  it  is  important 
to  establish  and  distribute  its  conceptual  and  technical  framework.  Without  developing  the  technological 
framework,  we  must  invest  in  developing  the  underlying  technology  each  time  we  develop  application 
systems.  The  technological  framework  for  network-based  distance  education  includes  4 component 
technologies:  modeling  of  educational  services,  courseware  authoring,  structuring  i storing  educationa 
information,  and  educational  user  interfacing.  Modeling  educational  services  means  establishing  theories 
and  guidelines  necessary  to  design  and  develop  various  network-based  education  systems.  Courseware 
authoring  technology  means  what  enables  users  to  easily  design  and  develop  various  types  of  network-based 
courseware.  Structuring  and  storing  educational  information  technology  means  what  is  necessary  to 
structure  educational  materials  upon  their  internal  and  external  characteristics  and  to  store  them  as  an 
integrated  educational  database  systems.  Educational  user  interfacing  technology  is  what  enable  users  to 

interact  vith  the  system  easily  in  retrieving  and  using  various  educational  materials  from  the  database 
systems. 

Only  the  systematic  development  of  the  above  component  technologies  can  exclude  redundant  investmen 
in  developing  them  for  each  application  system  and  guarantee  effective,  efficient,  and  appea’ ing  education 
systems  to  accomplish  the  role  of  future  education  for  society. 
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The  Use  of  Instructional  Development  Procedures  to  Create  Exhibits: 

A Survey  of  Major  American  Museums 

Museums  are  a major  American  pastime,  and  increasingly  they  are  also  big  business. 
There  are  more  than  6000  museums  in  this  country,  and  it  has  been  estimated  that  more 
people  attend  museums  each  year  than  attend  NFL  football  games.  It  has  also  been 
estimated  that  for  every  dollar  invested  in  museums,  they  generate  $30  in  revenue  for  the 
communities  in  which  they  are  located. 

Most  museums  also  acknowledge  that  they  have  an  educational  as  well  as  a 
recreational  purpose.  In  recent  years,  the  museums’  educational  missions  have  been 
highlighted  as  they  compete  for  public  funds  on  the  grounds  that  they  are  educational 
institutions.  Critics  have  pointed  out,  however,  that  museums  often  fail  to  educate.  B.  N. 
Lewis  stated  the  case  clearly  when  he  said,  “The  basic  problem  is  that  the  meaning  and 
significance  of  the  exhibits  tends  to  be  almost  completely  lost  on  the  generally  uninformed 
visitor.  An  exhibition  may  of  course  be  full  of  meaning  and  significance  to  the  visitor  who 
already  happens  to  know,  in  fairly  close  detail  what  the  subject  matter  is  all  about”  (p.  151). 

The  criticism  has  been  that  subject  matter  experts  create  museum  exhibits  that  assume 
a level  of  subject  matter  expertise  comparable  to  their  own,  when  in  fact  most  viewers  of  the 
exhibit  lack  the  prerequisite  knowledge  or  schema  to  comprehend  it.  On  the  other  hand, 
there  has  been  very  little  research  on  how  museum  exhibits  in  reality  are  created.  Who 
creates  these  exhibits?  For  whom  are  they  put  together?  Where  do  the  exhibit  creators  start? 
How  do  they  know  if  an  exhibit  is  successful?  Who,  if  anyone,  is  responsible  for  the 
educational  quality  of  the  exhibit?  Do  museums  use  instructional  systems  development  or 
parts  of  the  ISD  process  to  create  their  exhibits? 

The  purposes  of  the  research  described  in  this  paper  were  to  describe  through  rigorous, 
empirical  survey  research: 

1)  the  priority  major  American  museums  place  on  educational  exhibits. 

2)  the  extent  to  which  major  American  museums  use  instructional  development  (ID) 
practices  to  create  exhibits. 

A third  purpose  was  to  compare  the  results  of  this  investigation  with  a similar  one 
conducted  in  1986.  A complete  reporting  of  the  comparative  data  is  beyond  the  scope  of  this 
paper,  however.  When  the  longitudinal  analysis  is  completed,  the  results  will  be 
disseminated  through  national  conference  presentations  and  publication. 

Description  of  the  Study 

The  following  paragraphs  describe  the  sampling  strategy  that  was  used  in  the  research, 
the  types  of  museums  included,  the  survey  instrument,  and  the  response  received. 

Sampling  Strategy 

The  source  from  which  the  institutions  were  sampled  was  a database  supplied  by  the 
Institute  of  Museum  Services  (IMS).  The  sample  for  the  survey  included  museums  from 
all  the  various  IMS  categories.  The  sample  included  all  institutions  with  annual  budgets 
over  $2  million  that  applied  to  the  Institute  for  funding  during  the  five  years  preceding  the 
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study  (1990-1994),  and  those  with  an  annual  budget  less  than  $2  million  that  nevertheless 
represent  the  largest  budgets  within  their  institution  category.  The  latter  group  was 
included  because  some  types  of  museums  rarely  have  $2  million  budgets,  and  to  choose 
institutions  strictly  by  budget  size  would  result  in  the  omission  of  these  types  of  institutions 
from  the  study. 

This  sampling  strategy  identified  250  institutions  to  whom  the  research  instrument 
was  sent.  This  sample  approaches  population  data,  in  that  it  includes  nearly  all  of  the 
major  museums  in  the  country.  The  exceptions  would  be  private  museums  so  well  funded 
privately  that  they  rarely  apply  for  external  funds.  These  are  few  in  number. 

Types  of  Museums  Surveyed 

The  types  of  institutions  included  in  the  study  were:  aquarium,  arboretum/botanical 
garden,  art  museum,  children’s  museum,  historic  house,  history  museum,  natural 
history/anthropology  museum,  nature  center,  planetarium,  science/technology  museum, 
zoo,  specialized,  general,  and  “other.” 

The  Survey  Instrument 

The  instrument  used  in  the  study  included  59  close-ended  and  open-ended  questions. 
The  survey  was  organized  around  the  major  steps  in  the  instructional  systems 
development  process.  Definitions  of  these  steps  and  related  terminology  were  provided 
before  questions  about  the  use  of  these  steps  were  asked.  The  research  instrument  was 
created  by  the  authors  and  closely  paralleled  the  instrument  used  in  1986  by  Schlenk  (1987) 
to  assess  the  exhibit  development  process  at  that  time.  Major  modification  of  the  earlier 
instrument  was  precluded  because  an  important  purpose  of  thr  current  research  was  to 
compare  its  findings  with  those  of  the  previous  study.  Preserving  this  goal  meant  that  a 
number  of  questions  about  concepts  that  have  emerged  in  the  past  ten  years-for  example, 
constructivist  learning  environments-could  not  be  investigated  with  this  instrument. 

Response  to  the  Survey 

One  follow-up  postcard  reminder  was  sent  to  institutions  that  did  not  respond  within  the 
first  few  weeks  following  the  mailing  of  the  survey.  The  final  response  rate  was  quite 
good;  114  institutions  returned  completed  questionnaires  for  a final  response  rate  of  46%. 

Results  of  the  Study 

As  mentioned  above  the  survey  instrument  included  59  questions.  Therefore,  the 
results  are  voluminous.  This  paper  presents  an  overview  of  responses  to  some  of  the  most 
fundamental  questions  addressing  museum  priorities  and  the  exhibit  development 
process. 

Museum  Priorities 

Museums  rank  education  as  their  number  one  priority;  however,  exhibits  rank  a very 
close  second.  When  asked  to  rank  educational  activities,  museums  rank  exhibits  first. 
Museums  tend  to  believe,  however,  that  recreation  is  the  primary  reason  why  people  visit 
them. 
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Exhibit  Development  Time 


Museums  take  an  average  of  29.7  months  to  develop  a major  exhibit;  the  reported  range 
is  1 to  60  months.  Museums  take  an  average  of  10.1  months  to  develop  a minor  exhibit;  the 
reported  range  is  0.5  to  24  months. 

Use  of  Instructional  Systems  Development 

The  following  paragraphs  describe  the  extent  to  which  museums  reported  using  some  of 
the  essential  steps  in  the  instructional  development  process. 

Overall  use  of  ID.  Results  indicate  that  39%  of  museums  are  using  instructional 
development  to  create  their  exhibits.  Among  those  who  use  ID,  76%  consider  it  to  be  either 
“successful”  or  “very  successful.”  Of  museums  not  using  instructional  development,  22% 
report  that  they  are  not  familiar  with  the  ID  process. 

Needs  assessment.  Results  show  that  45.6%  of  museums  include  needs  assessment  as 
part  of  their  exhibit  development  process.  When  asked  how  audience  needs  are 
determined,  surveys  (68.4%)  and  trends  (67.5%)  are  reported  as  the  most  popular  tools, 
while  interviews  (42.9%)  are  also  widely  employed.  On  the  other  hand,  70%  of  museums  do 
not  use  audience  characteristics  data  when  selecting  exhibit  topics.  Interestingly,  67%  of 
museums  believe  they  are  responsive  to  their  audience's  needs. 

Use  of  objectives.  Only  20.7%  of  museums  always  have  written  instructional 
objectives  for  exhibits;  47.7%  sometimes  have  written  objectives.  Of  museums  that  use 
objectives  for  their  exhibits,  a mere  7.7%  have  objectives  stated  in  behavioral  terms. 

Formative  evaluation.  Formative  evaluation  is  one  of  the  better  accepted  steps  of  the  ID 
process  among  museums.  Responses  indicate  that  40.7%  of  museums  conduct  formative 
evaluation  of  exhibits.  However,  only  24.6%  of  museums  use  trial  runs  of  exhibits  as  a part 
of  their  exhibit  development  process. 

Summative  evaluation.  Museums  consider  large  crowds  as  the  main  indication  of  a 
successful  exhibit.  Further,  47%  of  museums  make  no  attempt  to  assess  what  their 
audiences  learn  from  their  exhibits. 

Instructional  developers.  Results  indicate  that  39%  of  museums  have  instructional 
developers  on  their  staffs.  In  addition,  36%  have  an  instructional  developer  working  as 
part  of  their  exhibit  development  team;  61%  said  they  should  have  instructional  developers 
on  their  exhibit  development  teams. 

Role  of  the  instructional  developer.  The  museum  field  appears  to  be  equally  split 
regarding  whether  an  instructional  developer  should  coordinate  the  exhibit  development 
process  - 47.7%  said  “yes”  while  46%  said  “no.” 

Responsibility  for  educational  effectiveness.  The  staff  member  most  likely  to  be 
charged  with  insuring  the  instructional  soundness  of  exhibits  is  a museum  administrator, 
not  an  educator  or  an  instructional  developer.  If  there  is  no  formal  position  to  insure  the 
instructional  soundness  of  exhibits,  this  duty  falls  to  someone  in  education  only  20%  of  the 
time. 


Preliminary  Conclusions 


The  data  collected  in  this  survey  allow  the  researchers  to  draw  several  preliminary 
conclusions  about  the  state  of  exhibit  development  in  major  American  Museums.  There 
are  significant  comparisons  to  be  made  with  the  original  1986  survey  that  will  be  published 
when  the  data  is  totally  analyzed  and  the  longitudinal  aspects  of  the  data  are  explored.  The 
conclusions  below  reflect  only  the  1996  survey  data. 

Museum  priorities.  Given  that  museums  rank  education  and  exhibits  as  their  first  two 
priorities  and  consider  exhibits  as  their  number  one  educational  activity,  the  conclusion 
might  be  drawn  that  the  priorities  museums  ranked  first  and  second  are  very  closely 
related;  they  see  both  as  education.  Historically  this  is  a major  change  from  the  original 
goals  of  most  museums  — to  collect,  preserve,  conduct  research,  and  publish  scholarly 
writings. 

Exhibit  development  teams.  Both  the  quantitative  and  qualitative  data  collected  with 
the  current  survey  indicate  that  many  museums  view  exhibit  development  as  a team  effort. 
However,  there  is  a diversity  of  opinion  about  who  should  be  a member  of  an  exhibit 
development  team  and  who  should  coordinate  the  work  of  such  teams.  A majority  of 
respondents  indicated  that  an  instructional  developer  should  be  included  as  a member  of 
an  exhibit  development  team.  Far  fewer  (though  a sizable  percentage)  indicated  that  an 
instructional  developer  should  coordinate  the  team  effort.  This  result  suggests  that  many 
museums  value  the  contribution  of  instructional  developers,  but  are  not  ready  to  turn  over 
control  of  the  exhibit  development  process  to  them. 

Use  of  instructional  development.  The  fact  that  nearly  40%  of  museums  use 
instructional  development  and  find  it  successful  would  indicate  that  instructional 
development  should  be  included  as  a basic  course  in  the  curriculum  of  the  museum  studies 
programs  offered  by  a number  of  universities  around  the  United  States.  This  fact  is 
underscored  by  the  fact  that  22%  of  the  museums  indicated  that  they  were  not  familiar  with 
the  ID  process. 

Evaluation  of  exhibits.  Evaluation  of  exhibits  is  conducted  through  a number  of  means 
by  museums.  While  formative  evaluation  of  exhibits  is  widely  used,  many  museums 
make  no  attempt  to  assess  what  their  audiences  learn  from  their  exhibits.  Yet  they  seem  to 
have  confidence  that  they  are  responsive  to  their  audiences*  needs.  The  conclusion  could 
be  drawn  that  museums  could  increase  their  use  of  formative  evaluation,  trial  runs,  and 
other  evaluation  strategies.  In  particular,  the  finding  that  many  museums  do  not  assess 
what  their  audiences  learn  from  their  exhibits  would  seem  to  be  inconsistent  with  the  high 
priorities  which  museums  ascribe  to  education  and  exhibits.  Museums  would  seem  to 
accept  evidence  of  their  effectiveness  that  instructional  developers  would  question. 

Recommendations  from  the  Survey  Results 

Exhibit  tonic  selection.  Museums  should  make  greater  efforts  to  consider  their 
audiences*  needs  when  selecting  and  developing  exhibit  topics. 

Instructional  development.  Teach  ID  as  an  integral  part  of  the  curriculum  of  museum 
studies  programs. 
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Increase  use  of  evaluation.  Museums  should  increase  their  use  of  evaluation 
strategies  to  insure  the  educational  effectiveness  of  their  exhibits.  All  museums  should 
attempt  to  learn  what,  if  anything,  visitors  are  learning  from  their  exhibits.  The 
assessment  of  what  their  audiences  learn  from  their  exhibits  should  be  conducted  in  light 
of  their  institution’s  mission  statement,  the  institution’s  statement  of  goals,  and  ideally  the 
specific  objectives  written  for  the  exhibit  being  evaluated. 

Successful  exhibits.  Museums  should  change  their  view  of  successful  exhibits.  Large 
crowds  are  the  major  indication  of  a successful  exhibit  for  most  museums.  Museums 
should  view  a successful  exhibit  as  an  educationally  effective  exhibit,  not  just  an  exhibit 
which  attracts  large  crowds.  Such  a view  would  be  in  keeping  with  museums’  assertion 
that  education  is  their  first  priority. 

Some  Final  Comments 

As  mentioned  above,  a primary  purpose  of  this  study  is  a comparison  of  current 
findings  to  the  results  of  the  1986  survey.  It  is  apparent  from  a cursory  comparison  of  the 
two  data  sets  that  there  has  been  significant  change  in  a number  of  areas  since  1986.  It  is 
also  apparent  from  an  instructional  development  viewpoint  that  many  exhibit  development 
practices  have  not  changed  for  the  better  since  1986. 

A quote  from  1982  provides  us  with  a snapshot  of  instructional  development  as  an 
emergent  “exhibit  development  technology”  at  that  time.  In  the  early  80’s  the  curators  at  the 
British  Museum  who  were  charged  with  installing  the  new  Hall  of  Human  Biology 
searched  for  ways  to  make  their  new  exhibit  more  educationally  effective;  they  knew  that 
their  previous  exhibits  had  not  been  as  educationally  effective  as  they  had  wished.  They 
searched  until  they  found  techniques  and  strategies  that  they  felt  would  make  their  exhibits 
more  educationally  effective.  They  were  pleased  with  their  “discoveries”  and  published 
The  Design  of  Educational  Exhibits  (Miles,  Alt,  Gosling,  Lewis,  & Tout,  1982)  to  spread  the 
good  word.  In  describing  the  systematic  process  they  discovered,  Miles  et  al.  (1982;  states: 

...  we  have  found  that  a lot  of  relevant  knowledge  does  in  fact  exist,  and  would 
assist  the  systematic  production  of  exhibitions  of  almost  any  kind.  However,  it 
does  not  for  the  most  part,  exist  in  organized  form  or  in  the  primary  museum 
literature.  Much  of  it  comes  from  “foreign”  fields  such  as  cybernetics,  psychology, 
educational  science  and  information  design  . . . many  exhibition  failures  are 
failures  of  knowledge  rather  than  failures  of  individuals  (Preface). 

Miles  never  appears  to  have  been  aware  that  the  field  of  instructional  design  incorporates 
all  of  the  “relevant  knowledge”  that  he  describes  as  coming  from  “foreign”  fields.  It  will 
be  interesting  and  informative  to  compare  the  results  of  the  1986  survey  with  the  current 
survey;  perhaps  exhibit  development  practice  has  improved  in  the  nearly  10  intervening 
years. 


The  authors  of  this  paper  would  like  to  express  their  gratitude  to  Marina  Tarshis  and  Mark 
Cody  who  worked  as  research  assistants  for  this  study.  Their  tireless  help  with  the  data 
analysis  did  much  to  help  the  authors  complete  this  paper. 
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ABSTRACT 


The  purpose  of  this  experiment  was  to  study  the  effects  of  media  types  and  camera  points  of 
view  in  self-instructional  materials  on  fifth  graders'  learning  of  a psychomotor  skill,  scout  knotwork. 
The  researchers  assigned  96  fifth  graders  to  six  groups  not  according  to  any  academic  standing. 
Subjects  learned  scout  knotwork  via  different  versions  of  self-instructional  materials  and  received  an 
immediate  performance  test.  There  were  two  independent  variables:  media  types  (videotape,  color 
photo  and  line-drawing)  and  camera  points  of  view  (subjective  and  objective  POVs). 

When  one  takes  a picture  or  shoots  video  footage  from  objective  camera  point  of  view,  one  puts 
the  camera  opposite  to  the  demonstrator,  with  the  lens  facing  the  demonstrator.  The  left-right 
orientation  on  the  screen  or  in  the  picture  is  opposite  to  that  of  the  actual  execution.  While  with 
subjective  camera  POV,  one  puts  the  camera  at  the  eye  level  or  slightly  to  the  side  of  the  demonstrator. 
Thus,  the  left-right  orientation  shown  on  the  screen  or  in  the  picture  is  the  same  as  the  actual  execution. 

Subjects  had  ten  minutes  to  leant  the  knotwork  from  one  of  the  six  versions  of  self-instnictional 
materials;  then  they  were  asked  to  perform  the  knotwork  without  any  reference.  There  was  a five- 
minute  time  limit  for  the  test.  Points  were  given  to  each  product  according  to  its  function  and  accuracy 
by  two  independent  raters.  Whether  subjects  finished  their  works  within  time  limit  and  how  long  it 
took  them  to  finish  were  also  recorded.  The  findings  were  as  follows. 

1.  There  was  a significant  effect  of  media  types  on  the  accuracy  of  the  knotwork.  Line-drawing  group 
performed  significantly  better  than  color  photo  group.  However  the  main  effect  of  camera  POVs  and 
the  interaction  effect  of  media  types  and  camera  POVs  on  accuracy  were  not  significant. 

2.  There  were  no  significant  effects  of  media  types  and  camera  POVs  on  the  function  of  the  knotwork. 

3.  There  were  no  significant  effects  of  media  types  and  camera  POVs  on  the  speed  and  the  amount  of 
time  subjects  need  to  finish  the  work. 

Based  upon  the  experiment  findings,  the  researchers  suggested  the  producers  of  self-instnictional 
materials  could  consider  to  use  line-drawing  to  present  knotwork  to  fifth  graders.  It  was  also  suggested 

that  future  researchers  could  consider  to  use  the  degree  of  realism  in  visuals  as  an  independent  variable 
for  further  investigation. 
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effects  of  media  types  and  camera  points  of  view 

IN  SELF-INSTRUCTIONAL  MATERIALS 
ON  FIFTH  GRADERS’  LEARNING  OF  PSYCHOMOTOR  SKILLS 


CHAPTER  1 INTRODUCTION 


stores  in  Taiwan,  one  finds  many  children'  craft  kits, 


BACKGROUND  OF  THE  STUDY 

In  shopping  around  toy  stores  or  stationary  : 
e.g.,  Legos,  Chinese  knotwork,  paper  -folding  kits,  model  car  assembly  kits,  among  many  others,  that 

accompanied  by  instructional  materials.  These  instructional  materials  take  different  forms, 


are 


including  videotapes  and  print  materials  (verb.1  explanations  plus  line-drawings  or  verba,  exp, .nations 
p,„s  pictures).  1.  is  expected  that  children  will  be  able  .0  follow  the  instructions  in  working  With  the 


craft  kit. 

Use  of  such  instructions  stimulates  research  interest  in  conducting  studies  to  investigate  e 
effect  of  media  types  on  children's  learning  of  psychontotor  skills.  According  .0  the  proper-use-of- 
media  suggestion,  there  should  be  a match  between  media  aBribu.es,  instructional  content,  and  learners. 
Most  of  the  craft  work  consists  of  dynamic  procedures  that  can  be  presented  well  in  vtdeotapes.  But  if 
children  wish  .0  have  on  overview  of  the  entire  procedure,  or  if  they  wish  to  refer  the  procedure  back 
and  forth,  then  print  materials  Seem  to  be  more  appropriate.  A great  many  instructional  mater 
consist  of  verbal  explanations  plus  line-drawings  due  to  their  low  production  cost.  Exactly  which 
media  type  best  facilitates  children's  learning  of  psychomotor  skills  such  as  craft  work? 

From  the  video  production  experience,  the  researchers  also  wish  to  compare  the  instruct, on.l 
effects  of  different  camera  points  of  view  (POVs).  When  one  takes  a picture  or  shoots  video  footage 
from  objective  earners  point  of  view,  one  puts  the  camera  opposite  to  the  demonstrator,  with  the  lens 
facing  the  demonstrator.  The  left-right  orientation  on  the  screen  or  in  the  picture  is  opposite  to  that  of 
the  actual  execution.  While  with  subjective  camera  POV,  one  puts  the  camera  a.  the  eye  level  or 
slightly  to  the  side  of  the  demonstrator.  Thus,  the  left-right  orientation  shown  on  the  screen  or  m the 
picture  is  the  same  as  the  actual  execution.  When  children  watch  video  or  read  print  materials  to  leant . 
psychomotor  skill,  such  as  scout  knotwork,  in  which  left-right  direction  makes  a difference,  which 

camera  POV  will  be  more  effective  instructionally? 

These  media  types  and  camera  POVs  are  the  variables  researchers  investigated  in  this  study. 
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RESEARCH  PURPOSE 

The  purposes  of  this  study  are  as  follows. 

1.  To  investigate  the  effect  of  media  type  on  children's  learning  of  psychomotor  skills. 

2.  To  investigate  the  effect  of  camera  POV  on  children's  learning  of  psychomotor  skills. 

3.  To  investigate  the  interaction  effect  of  media  type  and  camera  POV  on  children's  learning  of 
psychomotor  skills. 

RESEARCI 1 FRAMEWORK 
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SCOPE  OF  THIS  STUDY 

When  interpreting  results  of  this  study,  readers  are  encouraged  to  consider  the  following 
limitations. 

1.  Subjects  were  96  fifth  graders  enrolled  in  an  urban  public  elementary  school  in  northern  Taiwan. 
Their  IQ  score  was  considered  high  (PR=73)  among  children  of  this  age.  Thus,  the  findings  might 
not  apply  to  self-learners  of  other  ages  or  other  intelligent  levels. 

2.  The  psychomotor  skill  studied  was  scout  knotwork  which  requires  hand-eye  coordination,  but  not 
other  physical  skills.  Thus,  the  findings  might  not  apply  to  those  psychomotor  skills  that  require 
whole-body  coordination,  e.g.,  riding  a bicycle,  ice-skating,  etc.. 


CHAPTER  2 REVIEW  OF  LITERATURE 

Four  topics  are  addressed  in  this  chapter.  First,  a review  of  research  studies  on  realism  in  visuals 
is  presented;  second,  the  characteristics  of  media,  instructional  contents  and  learners  are  discussed; 
third,  related  important  literature  on  psychomotor  learning  is  reviewed  briefly;  finally,  research 
questions  and  hypotheses  are  listed. 

REALISM  IN  V SUALS 

It  is  said  that  "to  see  is  to  believe",  so  visuals  are  often  used  to  facilitate  learning.  This  could  be 
exemplified  at  all  levels  of  instructional  materials.  Photographs,  slides,  videotapes,  models,  actual 
objects  etc.  are  frequently  used  as  parts  of  various  instructional  strategies.  Research  on  visuals  can  be 
traced  back  to  the  1940s  and  1950s.  A wealth  of  theoretical  orientations  were  identified  in  this  period, 
such  as  Morris'  iconicity  theory  (1946),  Dale’s  cone  of  experience  (1946),  and  Carpenter's  sign 
similarity  hypothesis  (1953).  The  generic  idea  was  that  the  greater  number  and  the  more  realistic  the 
cues  in  the  learning  environment  are,  the  more  complete  the  learning  will  be.  Finn  (1953)  also 
suggested  that  the  more  realistic  or  lifelike  the  stimulus  in  instruction  was,  the  greater  the  probability  it 
had  for  facilitating  learning.  This  idea  is  similar  to  Osgood's  more  detachable-less  detachable 
continuum  (1953),  Gibson's  projective-conventional  continuum  (1954),  and  Knowlton's  transparency- 
opacity  continuum  (1964)  in  the  later.  The  main  thrust  of  these  theories  is  presented  in  Figure  2-1 . 
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Fig.  2-1 : Relationship  between  learning  achievement  and  visual  realism  (Realism  Theory) 

However,  Dwyer  (1978)  had  a totally  different  perspective.  He  argued  that  the  number  of  visual 
cues  in  the  learning  process  was  not  a guarantee  of  better  learning  results.  In  other  words,  there  was  an 
optimal  limit  on  the  number  of  visual  cues.  Before  reaching  learners'  upper  limit , the  relationship 
between  visual  cues  and  learning  outcome  was  gene-rally  similar  to  Morris,  et  al.  had  proposed,  but, 
when  the  number  of  visual  cues  exceeded  the  learner's  perception  load,  the  learning  results  would  then 
decline.  Therefore,  too  few  or  too  many  visual  cues  are  both  obstacles  to  effective  learning,  and  the 
relationship  between  learning  result  and  visual  realism  should  be  a curvilinear  one  (see  Figure  2-2). 
Before  Dwyer,  Miller  (1957),  Attneave  (1959),  and  Travers  et  al.  (1967)  also  claimed  that  irrelevant 
cues  would  interfere  with  the  learning  process.  Their  perspective  was  known  as  “Relevant  Cue 
Hypothesis”. 


Fig.  2-2:  Relationship  between  learning  result  and  visual  realism  (Relevant  Cue  Hypothesis) 

In  this  study,  the  researchers  selected  line-drawings,  photographs,  and  videotapes  as  the  three 
experimental  media  types.  According  to  Morris  (et  al.)'s  theory,  the  line-drawing  and  the  videotape 
groups  should  have  shown,  respectively,  the  lowest  and  the  highest  learning  efficiency  (see  Figure  2- 
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Fig.  2-3:  The  learning  efficiency  (based  on  Morris  et  al.) 

While  according  to  Dwyer  (1978),  realism  in  visual  illustrations  may  be  described  as  the  number 
of  stimuli  capable  of  conveying  information  to  students,  the  kind  of  stimuli  that  can  be  presented  in 
different  types,  amounts,  degrees  of  visual  realism  will  have  differing  effects  on  students'  learning 
achievements.  If  a teacher  gives  a highly  realistic  visual  stimulus  to  students,  students  might  fail  to  get 
the  basic  ideas,  because  students  are  probably  too  busy  watching  every  detail  and  forget  or  fail  to 
identify  the  primary  information.  Dwyer  called  this  situation  "the  scanning  syndrome".  After 
synthesizing  Dwyer’s  studies  from  the  1960s  to  the  1970s,  the  experimenters  found  it  difficult  to  derive 
hypotheses  for  the  present  study. 

The  researchers  found  only  few  studies  on  the  topic  of  psychomotor  learning.  Most  of  them 
focused  on  physical  training  (Baggett,  1983;  Ulrich,  1976),  or  on  disabled  learners4  skill  acquisition 
(Jeffries  & Pring,  1987;  Rodriguez,  1986).  Some  studies  mentioned  camera  angle  (Beverly  & Young, 
1978;  Bryski,  1988;  Eadie,  1991).  However,  they  investigated  the  relationship  between  viewers' 
perceptions  of  TV  news  anchors'  credibility  and  camera  angle  strategies,  which  was  outside  the  scope 
of  this  study.  Hozaki  (1987)  did  a study  on  the  effects  of  field  dependence/independence  and 
visualized  instruction  in  a paper-folding  lesson  on  performance  and  comprehension.  He  found  that 
videotape  group’s  learning  result  was  significantly  higher  than  a simple  line-drawing  group's,  and  that 
the  simple  line-drawing  group  showed  significantly  greater  learning  than  the  verbal  group.  However, 
Chou  (1990)  found  no  significant  difference  between  computer  graphic  presentation  format  and 
videotape  presentation  format  when  college  students  were  taught  to  evaluate  the  correct  position  for  the 
long  jump.  That  might  indicate  that  the  realism  of  visual  information  has  no  specific  relationship  with 
the  learner's  results. 

From  the  above  studies,  we  see  no  clear  indication  of  which  media  type  best  facilitates 
students1  learning  of  psychoinotor  skills.  The  researchers  also  question  whether  results  from  studies 
using  Americans  as  subjects  apply  to  a Taiwanese  environment.  How  can  instructors  use  visuals  for 
illustrations  effectively  and  efficiently?  This  is  the  main  concern  of  the  present  study. 

CHARACTERISTICS  OF  MEDIA,  INSTRUCTIONAL  CONTENTS  AND  LEARNERS 

Instructors  are  always  concerned  about  what  media  to  use,  what  contents  to  provide,  wiiat  kind 
of  students  to  teach,  and  what  kind  of  learning  effects  are  desired,  Salomon  (1983)  suggested  teachers 
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should  base  their  selections  on  media-attribute  analysis.  Following  is  the  attribute  comparison  of  line- 
drawing, photograph,  and  videotape. 


Still  Media 

Active  Media 

ivieuid  type 

Line-Drawing 

Photograph 

Videotape 

Appropriate 

Audience  with 

Audience  with 

Audience  with  TV 

Audience 

abstract-thinking 

concrete-thinking 

literacy  ability 

ability 

ability 

Appropriate 

•still 

•still 

•running/still 

Contents 

•need  to  be  observed 

♦need  to  be  observed 

•need  to  be  observed 

repeatedly 

repeatedly 

repeatedly 

•emphasize  main 

♦color 

•color 

character  by 

•continuing  process 

getting  rid  of 

•extend/shorten 

unnecessary 

time/space 

back/foreground 

Inappropriate 

•continuing  process 

♦continuing  process 

•contents  which  need 

Contents 

•concrete  contents 

•concrete  contents 

to  be  reordered 

•depth 

frequently 

•color 

•abstract  principles  or 

•present  depth 

rules 

and  colors 

% 

•long  text  in  length 

Figure  2-4:  Comparison  of  media  attributes 

| INSTRUCTIONAL  CONTENTS 

There  are  three  educational  objective  domains:  cognition,  affection  and  psychomotor  skills.  1 his 
study  concentrates  on  the  psychomotor  domain.  The  researchers  adopted  the  popular  psychomotor  skill 
taxonomy  (Harrow,  1972),  presented  briefly  below. 

Level  1 . Reflex  movements; 

Level  2.  Basic-fundamental  movements; 

Level  3.  Perceptual  abilities; 

Level  4.  Physical  abilities; 
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Level  5.  Skilled  movements;  and 
Level  6.  Non-discursive  communication. 

Each  of  the  above  six  can  be  divided  into  several  sub-levels.  For  instance.  Skilled  movement  can 
be  divided  into  three  sub-levels:  Simple  adaptive  skill.  Compound  skill  and  Complex  adaptive  skill. 
Every  sub-level  has  "Beginner",  "Intermediate",  "Advanced",  and  "Highly  skilled"  degrees. 

In  this  study,  scout  knotwork  is  the  target  skill.  It  requires  learners  to  coordinate  their  vision, 
thought  and  manual  dexterity  abilities  to  complete  the  specified  tasks  correctly  within  a limited  time. 
This  skill  is  a beginning  familiarity  one  on  the  Simple  adaptive  skill  sub-level  under  the  Skilled 
movements  level  in  Harrow’s  taxonomy. 

LEARNER’S  CHARACTERISTICS 

The  subjects  in  this  study  were  fifth  graders  in  a Taiwan  elementary  school.  According  to 
Piaget's  theory,  children  of  this  age  are  at  the  Concrete-operation  stage.  Most  of  these  children  are 
already  able  to  form  concrete  concepts  and  know  how  to  distinguish  things  by  their  features.  Salomon 
(1983)  and  Salomon  & Leigh  (1984)  found  that  children  usually  feel  TV  is  easier  to  understand  than 
print  media,  so,  they  invest  less  mental  effort  while  watching  TV  program.  This  is  an  important  reason 
why  children  have  lower  learning  results  after  watching  TV  but  have  higher  learning  results  after 
reading  contents  from  print.  The  researchers  also  interviewed  these  children’s  instructors  and  parents, 
who  mentioned  that  children  of  this  age  usually  don’t  like  to  do  things  that  are  considered  inappropriate 
to  their  gender.  For  example,  boys  don’t  like  to  learn  how  to  sew  dolls;  while  girls  don’t  like  to 
assemble  model  tanks.  Based  on  these  opinions,  the  authors  selected  a gender-free  skill  (i.e.,  scout 
knotwork)  for  this  study. 

RELATED  LITERATURES  ON  PSYCHOMOTOR  SKILL  LEARNING 

1.  Researcher:  Chou,  C.  (1990) 

Topic  : Effects  of  visual  information  presenta’  ion  mode  and  testing  mode  on  skill  analysis 
training  in  an  interactive  video  setting 
Subject  : 105  students  at  Ohio  State  University 

Finding  : The  degree  of  visual  realism  in  instructional  media  has  no  significant  effect  on  learning 
results 

2.  Researcher:  Hozaki,  N.  (1987) 

Topic  : The  effects  of  field  dcpendcnce/independence  and  visualized  instruction  in  a lesson  of 
origami,  paper  folding,  upon  performance  and  comprehension 
Subject  : 1 1 5 students  at  Ohio  State  University 

Finding  : 1 . Videotape  group’s  learning  results  are  significantly  better  than  simple  line-drawing 


group. 


2.  Simple  line-drawing  group’s  learning  results  is  significantly  better  than  print  group. 

3.  Researcher:  Baggett,  P.  (1983) 

Topic  : Learning  a procedure  from  multimedia  instructions:  The  effects  of  film  and  practice. 
Subject  :360  students  at  Colorado  University 

Finding  : 1 . Practice  group  had  the  highest  scores  in  an  immediate  posttest. 

2.  Practice-then-watch-video  group  had  the  highest  scores  in  a delayed  posttest. 

4.  Researcher:  Arnold,  T.  C.  & Dwyer,  F.  M.  (1975) 

Topic  : The  relationship  between  various  visual  effects  in  different  instructional  materials  and 
different  instructional  objectives. 

Subject  : 1 85  11  th-grade  students  in  USA 

Finding  : Subjects  learning  from  visual  instructional  materials  scored  significantly  higher  than 
counterparts  with  no  visuals. 

5.  Researcher:  Dwyer,  F.  M.  (1967,  1968,  1969a,  1969b,  1970,  1971a,  1971b,  1971c,  197 Id,  1971  e, 

1972,  1973a,  1973b,  1973c,  1975,  1976,  1977) 

Nature  of  Topics  : Serial  research  on  the  relationship  between  visual  realism  and  learning  results 
Finding  : 1.  The  realism  continuum  for  visual  illustrations  is  not  an  effective  predictor  of  learning 
efficiency  for  all  types  of  educational  objectives. 

2.  The  use  of  certain  types  of  visual  illustrations  to  facilitate  specific  types  of 
educational  objectives  significantly  improves  student  achievement. 

3.  All  types  of  visual  illustrations  are  not  equally  effective  in  facilitating  student 
achievement  of  different  educational  objectives. 

4.  The  use  of  visual  illustrations  designed  specifically  to  complement  printed  instruction 
does  not  automatically  improve  student  achievement. 

5.  For  the  teaching  of  specific  types  of  educational  objectives,  printed  instruction  without 
visualization  is  as  effective  as  visualized  instruction  for  certain  types  of  students. 

RESEARCH  QUESTIONS  AND  RESEARCH  HYPOTHESES 

The  authors  of  this  study  wished  to  answer  the  following  research  questions. 

Research  Question  U 1 : 

What  effect  do  media  type  and  camera  POV  have  on  the  function  of  subjects'  knotwork? 

Research  Hypothesis  #1 : 

Media  type  and  camera  POV  would  significantly  affect  the  function  of  subjects'  knotwork. 
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Research  Question  #2: 

What  effect  do  media  type  and  camera  POV  have  on  the  accuracy  of  subjects’  knotwork? 
Research  Hypothesis  #2: 

Media  type  and  camera  POV  would  significantly  affect  the  accuracy  of  subjects'  knotwork. 
Research  Question  #3: 

What  effect  do  media  type  and  camera  POV  have  on  subjects'  time-on-task? 

Research  Hypothesis  #3: 

Media  type  and  camera  POV  would  significantly  affect  subjects'  time-on-task. 


CHAPTER  3 METHODOLOGY 


SUBJECTS 

Subjects  were  96  fifth  graders  enrolled  in  an  urban  public  elementary  school  in  northern  Taiwan 
during  the  spring  of  1994.  Among  them,  48  were  boys  and  48  were  girls.  None  of  them  were  boy 
scouts  or  girl  scouts,  and  none  had  learned  scout  knotwork  or  Chinese  knotwork.  Descriptive  statistics 
of  subjects’  important  characteristics  were  reported  in  Table  3-1 . 


Table  3-1 

Group  Means  and  Standard  Deviations  of  Last  Semester's  Chinese  Language,  Arts  and  Crafts, 
Raven  Coloured  Progressive  Matrices  (CPM)  Score 


Chinese  Arts  & 

Language  Crafts  CPM  IQ 

Score  Score  Score 


Group 

n 

M 

SD 

M 

SD 

M 

SD 

Video  (Subjective  POV) 

16 

86.94 

4.97 

92.31 

1.89 

40.12 

8.66 

Video  (Objective  POV) 

16 

88.31 

4.59 

93.19 

1.80 

42.19 

9.96 

Color  photo  (Subjective  POV) 

16 

89.50 

3.54 

93.25 

2.18 

37.75 

8.54 

Color  photo  (Objective  POV) 

16 

90.56 

3.86 

93.50 

3.C3 

42.94 

4.37 

Line-drawing  (Subjective  POV) 

16 

90.69 

3.81 

94.31 

1.92 

43.69 

5.95 

Line-drawing  (Objective  POV) 

16 

88.44 

3.18 

93.31 

1.78 

43.00 

7.42 

Total 

96 

89.07 

4.15 

93.31 

2.17 

41.61 

7.79 

Analysis  of  Variance  (ANOVA)  indicated  that  there  was  no  significant  difference  among  these 
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six  groups’  mean  Chinese  Language  scores.  Arts  & Crafts  scores  and  CPM  IQ  scores;  F(5,  95)=2.0702, 
p=.0764;  F(5,  95)=1 .4302,  p=.221 1;  F(5,  95)=1.0319,  p=.4038.  The  researchers  thus  assumed  that 
these  groups  were  equal  in  terms  of  their  entry  behaviors  for  this  knotwork  lesson. 


INSTRUMENTS 
A.  Research  instruments 

Based  upon  the  two  independent  variables  in  this  experiment,  media  type  and  camera  POV,  the 
researchers  developed  the  following  six  versions  of  self-instructional  materials  on  scout  knotwork.  The 
task  consisted  of  two  kinds  of  knots,  which  combined  could  be  used  to  hang  up  a bucket  or  a bowel. 

1.  Subjective  POV  video 

2.  Objective  POV  video 

These  videotapes  were  four  minutes  and  forty  seconds  long.  There  were  Mandarin  Chinese  narration 
as  well  as  Chinese-language  subtitles.  The  demonstrator  was  a female  supervisor  of  the  scout  club  in 
the  experiment  elementary  school.  The  demonstrator’s  narration  was  recorded  on  the  shooting  spot. 

3.  Subjective  POV  color  photos 

4.  Objective  POV  color  photos 

There  were  eleven  color  photos  which  were  accompanied  by  their  verbal  explanation  perspectively. 
The  photos  emphasized  the  rope,  the  bucket,  and  the  demonstrator’s  hands,  though  other  things  (e.g., 
foreground,  background,  the  demonstrator)  still  could  been  seen  partly. 

5.  Subjective  POV  line-drawings 

6.  Objective  POV  line-drawings 

There  were  eleven  line-drawings  which  were  accompanied  by  their  verbal  explanation  perspectively. 
Only  the  rope  and  the  bucket  were  presented  in  the  line-drawings;  the  demonstrator,  the  foreground 
and  the  background  were  omitted. 

The  verbal  explanation  in  these  six  group  was  identical  in  terms  of  its  contents.  However,  it 
differed  in  terms  of  the  presenting  format.  In  the  two  video  versions,  the  verbal  explanation  was 
presented  in  the  formats  of  narration  and  subtitle.  In  the  photos  and  line-drawings  versions,  the  verbal 
explanation  was  presented  in  print. 

B.  Evaluation  Instrument 

The  researchers  adopted  the  product  evaluation  method  to  assess  subjects’  learning  achievement. 
They  developed  a checklist  with  the  consultation  from  children  scout  supervisor  in  the  elementary 
school.  According  to  the  checklist,  the  maximum  point  was  six,  one  point  for  the  function  of  the  knot- 
work  (i.e.,  could  be  used  to  hang  up  a bucket  or  a bowel  successfully),  the  other  Five  points  for  the 
accuracy  of  the  knotwork. 

To  avoid  experimenter  bias,  the  researchers  had  two  independent  raters  who  were  blind  to  the 
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purpose  of  this  experiment  evaluated  subjects’  knotwork,  the  Pearson  Product  Moment  Correlation 
coefficients  were  1 .000  (for  the  function  score)  and  .993 1 (for  the  accuracy  score)  respectively 
(P<  -01). 


PROCEDURES 

The  study  adopted  factorial  experiment  design.  There  were  two  independent  variables.  One  was 
media  type,  which  included  videotape,  color  photo  and  line-drawing;  the  other  was  camera  POV, 
which  included  subjective  and  objective  camera  POVs. 

CAMERA  POINTS  OF  VIEW 


SUBJECTIVE 

OBJECTIVE 

VIDEO 

Group  1 

Group  2 

COLOR 

PHOTO 

Group  3 

Group  4 

LINE- 

DRAWING 

Group  5 

Group  6 

S 


Figure  3-1:  Experiment  design 

Ninety  six  subjects  were  assigned  to  these  six  groups  not  according  to  any  academic  standing. 
There  were  sixteen  subjects  in  each  group.  Pretest  was  not  administered  since  the  researchers  assumed 
these  subjects  were  equal  in  terms  of  their  entry  behaviors  for  the  following  reasons. 

1 . None  of  these  subjects  were  boy  scouts  or  girl  scouts. 

2.  None  of  these  subjects  had  learned  scout  knotwork  or  Chinese  knotwork. 

3.  The  preliminary  Statistical  analyses  indicated  these  subjects  were  equal  on  their  Chinese-languagc 
scores.  Arts  and  Crafts  scores,  and  CPM  IQ  scores. 

The  experiment  was  conducted  using  lunch  recess  hours  during  March,  1994.  The  procedure 
included  an  opening  announcement  which  informed  subjects  about  the  experimental  procedure,  a self- 
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learning  period  (ten  minutes)  and  an  immediate  posttest  (five  minutes).  The  total  length  of  the 
experiment  was  about  thirty  minutes.  During  the  self-learning  period,  subjects  were  given  the  self- 
instructional  materials,  a rope,  and  a bucket.  They  could  decide  how  they  wanted  to  use  them  to 
facilitate  their  learning.  Some  children  read  the  photo/line-drawing  brochure  or  watched  the  video  first, 
then  tried  to  perform  the  knotwork.  But  the  majority  of  the  subjects  tried  to  perform  the  knotwork 
while  going  through  self-instructional  materials. 

To  avoid  experimenter  bias,  the  researchers  had  a teacher  from  the  elementary  school  act  as  the 
experimenter,  and  two  independent  raters  evaluate  subjects’  products.  All  of  them  were  blind  to  the 
purpose  of  this  experiment.  The  acting  experimenter  knew  some  of  the  subjects,  while  the  raters  didn’t 
know  any  of  the  subjects.  The  authors  of  this  study,  who  didn’t  know  any  of  the  subjects,  were  present 
all  the  time  to  ensure  the  experiment  was  conducted  according  to  the  procedure. 

STATISTICAL  ANALYSIS 

Arithmetic  mean,  standard  deviation,  Pearson  product-moment  correlation,  two-way  analysis  of 
variance,  Tukey's  HSD  post  hoc,  as  well  as  the  test  of  homogeneity  of  proportions  in  chi-square  test 
were  performed  on  the  data  collected  from  the  experiment. 


CHAPTER  4 RESULTS 


Research  Question  # 1 : 

What  effect  do  media  type  and  camera  POV  have  on  the  function  of  subjects’  knotwork? 

Finding  # 1 : 

The  test  of  homogeneity  of  proportions  in  chi-square  test  was  performed  on  the  function  score  of 
subjects'  knotwork.  It  was  found  that  there  was  no  significant  difference  among  media  types  (chi- 
square  value=  1.0 1 1 4,  p>.05)  and  camera  POVs  (chi-square  value  = . 1 364,  p>.05)  on  percentages  of 
the  knot-work  which  could  be  used  to  hang  up  a bucket  successfully. 

Discussion: 

The  experimenters  felt  that  these  non-significant  differences  might  due  to  the  loose  grading 
policy,  i.e.,  as  long  as  the  knot  could  be  used  to  hang  up  a bucket  or  a bowel,  the  graders  considered 
the  knot  be  a functional  one  and  granted  the  function  score. 

Research  Question  U2: 

What  effect  do  media  type  and  camera  POV  have  on  the  accuracy  of  subjects'  knotwork? 

Finding  it 2 : 

Analysis  of  Variance  was  performed  on  the  accuracy  score  of  the  knotwork.  It  was  found  that 
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media  type  had  a significant  effect  on  the  accuracy  of  the  knotwork  (F=4.835t  p=.010).  Tukey's  HSD 
post  hoc  indicated  that  line-drawing  group  performed  significantly  better  than  the  color  photo  group 
(p=.010).  However  the  main  effect  of  camera  POV  and  the  interaction  effect  of  media  type  and  camera 
POV  on  accuracy  was  not  significant  (F=.150,  p=.699;  F—  1.566,  p^.215).  See  tables  4-1, 4-2  and  4-3. 

Table  4-1: 

Summary  Table  of  Analysis  of  Variance  on  Accuracy  Scores 
Sources  of 


Variation 

d.f. 

SS 

MS 

F 

P 

Media  type 

2 

11.314 

5.657 

4.835 

.010** 

Camera  POV 

1 

.176 

.176 

.150 

.699 

Interaction 

2 

3.665 

1.832 

1,566 

.215 

Error 

85 

99.457 

1.170 

Total 

90 

114.681 

1.274 

Table  4-2: 

Descriptive  Statistics  of  Different  Media  Type  on  Accuracy  Scores 


Media  Type 

n 

M 

SD 

Min. 

Max. 

1.  Videotape 

32 

2.78 

1.34 

0 

5 

2.  Color  Photo 

32 

2.28 

1.08 

0 

5 

3.  Line-drawing 

32 

2.84 

1.22 

0 

5 

Table  4-3: 

Descriptive  Statistics  of  Different  Camera  POV  on  Accuracy  Scores 
Camera  POV  n M SD  Min.  Max. 


1 . Subjective  POV  48  2.48  1.41  0 5 

2.  Objective  POV  48  2.79  1.01  0 5 


0 
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Discussion: 

This  finding  (line-drawing  group  performed  significantly  better  than  color  photo  group)  didn't 
support  the  realism  theories  (Morris,  1946;  Dale,  1946;  Carpenter,  1953;  Finn,  1953;  Osgood,  1953; 
Gibson,  1954;  Knowlton,  1964);  nor  did  it  correspond  with  the  finding  reported  by  Hozaki  (1987).  In 
his  dissertation,  Hozaki  investigated  the  effects  of  different  type  of  visual  instruction  on  paper-folding 
performance.  He  reported  that  the  videotape  group  performed  significantly  better  than  the  line-drawing 
group;  and  that  the  line-drawing  group  performed  significantly  better  than  the  verbal  group.  All 
differences  reached  .001  significance  level.  However,  the  researcher  believed  the  grading  procedures 
were  arguable,  since  Hozaki  himself  was  one  of  the  two  graders,  experimenter  bias  might  have  been 
presented;  and  he  did  not  report  the  inter-rater  reliability. 

This  finding  (line-drawing  group  performed  significantly  better  than  color  photo  group)  supported 
Attneave  (1959),  Dwyer  (1976),  Miller  (1957)  and  Travers  et.  al.  (1967).  They  all  suggested  that 
learning  achievement  declined  when  irrelevant  cues  and  important  instructional  cues  were  presented 
since  it  created  disturbance.  Flemming  (1970)  also  suggested  instructional  designers  present  only  those 
absolutely  necessary  instructional  cues  to  avoid  wasting  perceptual  capacity.  In  this  study,  only  the 
scout  rope  and  the  bucket  were  presented  in  the  line-drawing  version  of  the  self-instructional  materials; 
the  demonstrator,  the  foreground  and  the  background  were  all  omitted.  While  subjects  in  the  photo 
group  might  be  attracted  by  the  irrelevant  cues,  such  as  the  body,  the  clothes  of  the  demonstrator,  and 
the  background.  Also,  Travers  et  al.  (1964)  proposed  an  interesting  vie  vpoint,  i.e.,  visual  information 
was  stored  in  a form  that  resembled  the  format  of  line-drawing  in  the  immory..Thus  material  presented 
in  a line-drawing  format  is  easier  for  human  beings  to  organize,  process  and  memorize.  Our  findings 
also  support  this  viewpoint.. 

Our  other  finding,  that  camera  POV  had  non  significant  effect,  could  be  explained  by  the 
experimental  observation  that  almost  all  the  subjects  who  were  assigned  objective  POV  materials  tried 
to  simulate  the  subjective  POV  by  turning  their  bodies  slightly  to  more  easily  follow  instructions  with 
left-right  orientations  (e.g.,  "Put  the  rope  in  your  left  hand  on  the  top  of  the  rope  in  your  right  hand".). 

Research  Question  U3: 

What  effect  do  media  type  and  camera  POV  have  on  subjects'  time-on-task? 

Finding  #3: 

Subjects  were  classified  into  two  categories,  those  who  finished  the  knotwoik  within  five-minute 
time  limit  and  those  who  didn’t.  The  test  of  homogeneity  of  proportions  in  the  chi-square  test  was 
performed  to  analyze  the  data.  It  was  found  that  there  was  no  significant  difference  among  media  types 
(chi-square  value=  1 .0292,  p>.05)  and  camera  POVs  (chi-square  value  = .8440,  p>.05)  on  percentages 
of  subjects  finished  within  the  time  limit. 

The  researcher  then  performed  ANOVA  on  time  (in  seconds)  subjects  used  to  finish  their  tasks. 
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Those  who  didn't  finish  within  the  time  limit  were  eliminated  from  this  analysis.  Analysis  results 
indicate  that  there  were  no  significant  differences  among  media  type  (F=2.386,  p=  098),  camera  POV 
(F=l  .630,  p=.205)  and  interaction  (F=.142,  p=.868).  See  tables  4-4,  4-5  and  4-6. 

Table4-4: 

Summary  Table  of  Analysis  of  Variance  in  time-on-task  (in  seconds) 

Sources  of 


Variation 

d.f. 

ss 

MS 

F 

P 

Media  type 

2 

23671.263 

1 1835.831 

2.386 

.098 

Camera  POV 

1 

8086.560 

8086.560 

1.630 

.205 

Interaction 

2 

1411.230 

705.615 

.142 

.868 

Error 

85 

421721.189 

4961.426 

Total 

90 

455325.033 

Table_4-5: 

Descriptive  Statistics  on  Different  Media  Type  in  Time-on-task  (in  seconds) 


Media  Type 

n 

M 

SD 

Min. 

Max. 

1.  Video  Tape 

30 

131.90 

77.12 

40 

300 

2.  Color  Photo 

32 

152.97 

76.16 

37 

300 

3.  Line-drawing 

29 

113.24 

53.10 

39 

300 

Table  4-6. 

Descriptive  Statistics  on  Different  Camera  POVs  on  Time-on-task  (in  seconds) 
Camera  POV  n M SD  Min.  Max. 


1 . Subjective  POV  44  143.36  67.78  39  300 

2.  Objective  POV  47  124.00  73.61  37  300 
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Discussion: 

Five  of  the  96  subjects  failed  to  finish  the  knotwork  within  the  five-minute  time  limit.  Two  of 
these  five  subjects  belonged  to  the  videotape  group  and  the  other  three  belonged  to  the  line-drawing 
group.  Four  of  these  five  subjects  belonged  to  the  subjective  POV  group,  while  the  other  one  belonged 
to  the  objective  POV  group.  They  were  all  excluded  from  the  ANOVA  analysis.  Had  the  researcher 
requested  subjects  to  finish  the  knotwork  regardless  of  the  time  limit,  it  would  have  enlarged  the  mean 
difference  between  subjective  and  objective  POV  groups,  and  would  probably  have  resulted  in 
significant  differences  in  time.  However,  this  inclusion  would  have  reduced  the  mean  difference 
among  video,  photo  and  line-drawing  groups. 


CHAPTER  5 SUMMARY  AND  RECOMMENDATION 


SUMMARY 

In  order  to  investigate  the  effects  of  media  type  and  camera  POV  in  self-instructional  materials 
on  children's  learning  of  scout  knotwork,  the  researchers  assigned  96  fifth  graders  to  six  groups 
according  to  no  academic  standing.  Subjects  learned  scout  knotwork  via  different  versions  of  self- 
instructional  materials  and  received  an  immediate  performance  test.  The  six  groups  used: 

1 . subjective  POV  video, 

2.  objective  POV  video, 

3.  subjective  POV  color  photos, 

4.  objective  POV  color  photos, 

5.  subjective  POV  line-drawings,  and 

6.  Objective  POV  line-drawings. 

Subjects  had  ten  minutes  to  learn  the  knotwork  from  one  of  the  six  versions  of  self-instructional 
materials;  they  were  then  asked  to  perform  the  knotwork  without  any  reference  to  self-instructional 
materials.  There  was  a five-minute  time  limit  for  the  test.  Points  were  given  for  each  product  according 
to  its  function  and  accuracy  by  two  independent  raters.  Whether  subjects  finished  their  tasks  within  the 
time  limit,  and  how  long  it  took  them  to  finish,  were  also  recorded.  The  findings  are  as  follows. 

1 . There  was  a significant  effect  of  media  types  on  the  accuiacy  of  the  knotwork.  The  line-drawing 
group  performed  significantly  better  than  the  color  photo  group.  However  the  effect  of  camera  POV 
and  the  interaction  effect  of  media  type  and  camera  POV  on  accuracy  were  not  significant. 

2.  Media  type  and  camera  POV  had  no  significant  effects  on  the  function  of  the  knotwork. 

3.  Media  types  and  camera  POV  had  no  significant  effects  on  the  speed  and  the  amount  of  time 
suojccts  need  to  finish  the  work. 
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RECOMMENDATION 


Based  upon  these  experimental  findings,  the  researchers  suggest  that  producers  of  self- 
instructional  materials  consider  using  line-drawing  to  present  knotwork  to  fifth  graders.  It  is  also 
suggested  that  future  researchers  consider  using  the  degree  of  realism  in  visuals  as  an  independent 
variable  for  further  investigation. 
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